diff options
| -rw-r--r-- | INSTALL.md | 120 | ||||
| -rw-r--r-- | MAINTAIN.md | 58 | ||||
| -rw-r--r-- | README.md | 92 |
3 files changed, 203 insertions, 67 deletions
@@ -6,19 +6,33 @@ Copyright © 2016 Paul Garlick <[pgarlick@tourbillion-technology.com](mailto:pga notice and this notice are preserved. This file is offered as-is, without any warranty. -*TWEED* makes use of three package managers. As a consequence, there are three parts to the installation process. The parts can be followed in any order. +*TWEED* makes use of three package managers. As a consequence, there are three +parts to the installation process. The parts can be followed in any order. -As a preliminary step, download the *TWEED* repository. Use either the 'git clone' method or download the tar file. There are three documentation files, a LICENSE file, an AUTHORS file, a script and three configuration files, one for each package manager. +As a preliminary step, download the *TWEED* repository. Use either the 'git +clone' method or download the tar file. There are three documentation files, a +LICENSE file, an AUTHORS file, a script and three configuration files, one for +each package manager. Part 1: GNU Guix packages ------------------------- -GNU Guix is a transactional package manager. It can be used to install a complete GNU/Linux operating system. Transactional here means that all upgrades are performed as a sequence of atomic operations; the upgrade either completes fully or not at all. This provides protection against power loss in the middle of an upgrade; the system can reboot to its previous configuration. +GNU Guix is a transactional package manager. It can be used to install a +complete GNU/Linux operating system. Transactional here means that all +upgrades are performed as a sequence of atomic operations; the upgrade either +completes fully or not at all. This provides protection against power loss in +the middle of an upgrade; the system can reboot to its previous configuration. -GNU Guix may also be used to install packages on a computer running a different operating system. The GNU Guix package files are installed alongside the system files. This is how GNU Guix is used in *TWEED*. +GNU Guix may also be used to install packages on a computer running a different +operating system. The GNU Guix package files are installed alongside the +system files. This is how GNU Guix is used in *TWEED*. -First, install GNU Guix; follow the installation instructions in the [GNU Guix manual](https://www.gnu.org/software/guix/manual/html_node/Installation.html#Installation). There are two options; the binary installation method is the easier of the two. +First, install GNU Guix; follow the installation instructions in the [GNU Guix +manual](https://www.gnu.org/software/guix/manual/html_node/Installation.html#Installation). +There are two options; the binary installation method is the easier of the two. -Second, set the environment variables to access to the ``guix`` executable. This can be done using a shell function. For a *bash* shell, for example, the following function can be defined in the .bashrc file: +Second, set the environment variables to access to the ``guix`` executable. +This can be done using a shell function. For a *bash* shell, for example, the +following function can be defined in the .bashrc file: startGuix () { # set Guix environment variables @@ -29,7 +43,8 @@ Second, set the environment variables to access to the ``guix`` executable. Thi GUIX_LOCPATH=$HOME/.guix-profile/lib/locale } -To continue, start a new shell (or ``source`` the modified .bashrc file), move to the download directory and type: +To continue, start a new shell (or ``source`` the modified .bashrc file), move +to the download directory and type: $ startGuix @@ -37,21 +52,33 @@ Then, to install the *TWEED* packages, type. $ guix package --manifest=tweed.guix.scm -Aliases can be defined to start the applications. For example, for the grid generator Gmsh, the following alias sets ``FONTCONFIG_PATH`` to the correct location before executing the ``gmsh`` command: +Aliases can be defined to start the applications. For example, for the grid +generator Gmsh, the following alias sets ``FONTCONFIG_PATH`` to the correct +location before executing the ``gmsh`` command: $ alias gmsh="FONTCONFIG_PATH=$HOME/.guix-profile/etc/fonts gmsh" Part 2: Nix packages -------------------- +The Nix package manager is related to GNU Guix. Nix is the forerunner to GNU +Guix and provides some of the code on which GNU Guix is based. Many +characteristics are common to both package managers. For example, Nix forms +the basis for NixOS, a full GNU/Linux operating system. Nix upgrades are also +atomic. -The Nix package manager is related to GNU Guix. Nix is the forerunner to GNU Guix and provides some of the code on which GNU Guix is based. Many characteristics are common to both package managers. For example, Nix forms the basis for NixOS, a full GNU/Linux operating system. Nix upgrades are also atomic. +The Nix package manager can be installed alongside an operating system other +than NixOS. It does not conflict with GNU Guix; the two package managers store +their files separately. -The Nix package manager can be installed alongside an operating system other than NixOS. It does not conflict with GNU Guix; the two package managers store their files separately. +To install the Nix components of *TWEED*, first install Nix. Follow the +installation procedure shown on the [Nix +website](http://nixos.org/nix/download.html). -To install the Nix components of *TWEED*, first install Nix. Follow the installation procedure shown on the [Nix website](http://nixos.org/nix/download.html). - -By default, Nix modifies the users .bashrc file to automatically set up the Nix environment for new invocations of the shell. Should the user prefer, these lines can be commented out and a shell function defined to set up the environment: +By default, Nix modifies the users .bashrc file to automatically set up the Nix +environment for new invocations of the shell. Should the user prefer, these +lines can be commented out and a shell function defined to set up the +environment: startNix () { # set Nix environment variables @@ -60,15 +87,24 @@ By default, Nix modifies the users .bashrc file to automatically set up the Nix fi } -The user then has control over whether and when the Nix environment is set up. If the function is defined, then to use the Nix executables, or later the applications installed with Nix, first type: +The user then has control over whether and when the Nix environment is set up. +If the function is defined, then to use the Nix executables, or later the +applications installed with Nix, first type: $ startNix -Nix uses the ``nix-env`` command to control the package environment. To install the *TWEED* packages, use the following command: +Nix uses the ``nix-env`` command to control the package environment. To +install the *TWEED* packages, use the following command: $ nix-env -f tweed.nix -i -Nix applications that use 3D graphics acceleration need help in finding the system libraries and drivers. The paths vary according to operating system and graphics card. For a CentOS 6 system, for example, with an Intel graphics card six extra paths need to be defined. This can be accomplished with an alias or a shell function. Using the shell function approach for the CentOS example, the following lines added to the .bashrc file provide a function that sets the necessary environment variables and runs the FreeCAD application: +Nix applications that use 3D graphics acceleration need help in finding the +system libraries and drivers. The paths vary according to operating system and +graphics card. For a CentOS 6 system, for example, with an Intel graphics card +six extra paths need to be defined. This can be accomplished with an alias or +a shell function. Using the shell function approach for the CentOS example, +the following lines added to the .bashrc file provide a function that sets the +necessary environment variables and runs the FreeCAD application: freecad () { FONTCONFIG_PATH="$HOME/.nix-profile/etc/fonts/fonts.conf" \ @@ -85,43 +121,69 @@ Nix applications that use 3D graphics acceleration need help in finding the syst Part 3: HashDist packages ------------------------- - -HashDist is a package manager written in the Python programming language. It is similar to GNU Guix and Nix in that each item in the software store is identified by a cryptographic hash. By default, HashDist creates a subdirectory in the user's home directory. Packages installed by HashDist are stored in a hierarchy rooted at this subdirectory. - -There are two parts to a HashDist installation; installing the HashDist software and installing a HashStack, a set of instructions for building packages. The users HashDist profile defines which exact packages to build and install. In our case we will be using the ``tweed.hashdist.yaml`` file. - -First, install HashDist. Requirements for HashDist are listed on the [FEniCS Project website](https://fenicsproject.org/download/installation_using_hashdist.html). One of the requirements is the Git version control system. This is used for the initial installation. In a suitable subdirectory of the user's home directory type the following command: +HashDist is a package manager written in the Python programming language. It +is similar to GNU Guix and Nix in that each item in the software store is +identified by a cryptographic hash. By default, HashDist creates a +subdirectory in the user's home directory. Packages installed by HashDist are +stored in a hierarchy rooted at this subdirectory. + +There are two parts to a HashDist installation; installing the HashDist +software and installing a HashStack, a set of instructions for building +packages. The users HashDist profile defines which exact packages to build and +install. In our case we will be using the ``tweed.hashdist.yaml`` file. + +First, install HashDist. Requirements for HashDist are listed on the [FEniCS +Project +website](https://fenicsproject.org/download/installation_using_hashdist.html). +One of the requirements is the Git version control system. This is used for +the initial installation. In a suitable subdirectory of the user's home +directory type the following command: $ git clone https://github.com/hashdist/hashdist.git -This installs the ``hit`` command, which is used to build and install packages. To make this command available add the HashDist ``bin`` directory to the user's $PATH. For example, if HashDist has been installed in a subdirectory named ``repos`` then add the following lines to the user's .profile file +This installs the ``hit`` command, which is used to build and install packages. +To make this command available add the HashDist ``bin`` directory to the user's +$PATH. For example, if HashDist has been installed in a subdirectory named +``repos`` then add the following lines to the user's .profile file if [ -d "$HOME/repos/hashdist/bin" ]; then PATH="$HOME/repos/hashdist/bin:$PATH" fi -Next, install the HashStack. A FEniCS fork of the HashDist HashStack is available. It is kept up-to-date with the latest FEniCS developments. In a suitable directory (``$HOME/repos`` again, for example), install the FEniCS version with the command: +Next, install the HashStack. A FEniCS fork of the HashDist HashStack is +available. It is kept up-to-date with the latest FEniCS developments. In a +suitable directory (``$HOME/repos`` again, for example), install the FEniCS +version with the command: $ git clone https://github.com/FEniCS/hashstack.git -Start a new shell or Initialize HashDist to create the store root directory (``~/.hashdist`` by default): +Start a new shell or Initialize HashDist to create the store root directory +(``~/.hashdist`` by default): $ hit init-home -To prepare to install the packages for *TWEED*, change to the HashStack directory and copy the *TWEED* profile: +To prepare to install the packages for *TWEED*, change to the HashStack +directory and copy the *TWEED* profile: $ cd $HOME/repos/hashstack $ cp .../tweed.hashsdist.yaml default.yaml -Parts of the build process may be executed in parallel. For a system with N cores, use the command: +Parts of the build process may be executed in parallel. For a system with N +cores, use the command: $ hit build -jM -where M is less than N. A symbolic link named ``default`` is created in the HashStack directory. This link points to the newly created HashDist profile. The script ``tweed.hashdist.post-install`` is provided to read the link and create a new file containing the definitions of the necessary environment variables. From the HashStack directory run the command: +where M is less than N. A symbolic link named ``default`` is created in the +HashStack directory. This link points to the newly created HashDist profile. +The script ``tweed.hashdist.post-install`` is provided to read the link and +create a new file containing the definitions of the necessary environment +variables. From the HashStack directory run the command: $ sh tweed.hashdist.post-install -The file generated, named ``hashdist.sh,`` is put in a new directory ``.hashdist-profile``. Create a shell function to read and execute the commands in this file. Store the function in the .bashrc file. +The file generated, named ``hashdist.sh,`` is put in a new directory +``.hashdist-profile``. Create a shell function to read and execute the +commands in this file. Store the function in the .bashrc file. startHashDist () { # set HashDist environment variables diff --git a/MAINTAIN.md b/MAINTAIN.md index d378832..1e5ca62 100644 --- a/MAINTAIN.md +++ b/MAINTAIN.md @@ -1,15 +1,29 @@ # Tweed.MAINTAIN -*TWEED* applications are upgraded by updating the respective package manager. The update procedure installs new versions of the applications and any new versions of dependencies. The update frequency can be determined by keeping an eye out for new application versions. If there are of the order of ten version upgrades per year, for example, then a monthly upgrade cycle would be a good way to keep up-to-date. - -All three package managers use a hash value as part of the pathname for an application. Since all versions of an application have a different hash value there is no conflict between older and newer versions. Older versions are retained in the store by default. It is possible to immediately switch to any version that has previously been installed. - -Each of the three package managers has its own update procedure. To upgrade any given application it is only necessary to update one package manager. Sometimes an application has not been upgraded but some of its dependencies have been. It is up to the user to decide whether or not to update the package manager in this case. It is often acceptable to skip the dependency updates and wait for an upgrade to the application itself. - -The update procedures are outlined below. Each package manager also offers a way to delete unused files. This is termed *garbage collection*. +*TWEED* applications are upgraded by updating the respective package manager. +The update procedure installs new versions of the applications and any new +versions of dependencies. The update frequency can be determined by keeping an +eye out for new application versions. If there are of the order of ten version +upgrades per year, for example, then a monthly upgrade cycle would be a good +way to keep up-to-date. + +All three package managers use a hash value as part of the pathname for an +application. Since all versions of an application have a different hash value +there is no conflict between older and newer versions. Older versions are +retained in the store by default. It is possible to immediately switch to any +version that has previously been installed. + +Each of the three package managers has its own update procedure. To upgrade +any given application it is only necessary to update one package manager. +Sometimes an application has not been upgraded but some of its dependencies +have been. It is up to the user to decide whether or not to update the package +manager in this case. It is often acceptable to skip the dependency updates +and wait for an upgrade to the application itself. + +The update procedures are outlined below. Each package manager also offers a +way to delete unused files. This is termed *garbage collection*. Part 1: GNU Guix update procedure --------------------------------- - The command to update the GNU Guix installation is: $ guix package --upgrade @@ -22,15 +36,16 @@ The command to delete previous application versions is: $ guix package --delete-generations -The command to download the latest version of GNU Guix, with its list of the new package versions, is: +The command to download the latest version of GNU Guix, with its list of the +new package versions, is: $ guix pull -More information on these commands is avaialable in the [GNU Guix manual](https://www.gnu.org/software/guix/manual/html_node/index.html). +More information on these commands is avaialable in the [GNU Guix +manual](https://www.gnu.org/software/guix/manual/html_node/index.html). Part 2: Nix update procedure ---------------------------- - The command to update the Nix installation is: $ nix-env --upgrade @@ -47,20 +62,25 @@ The command to download the latest list of the packages is: $ nix-channel --update -More information on these commands is avaialable in the [Nix manual](http://nixos.org/nix/manual/). +More information on these commands is avaialable in the [Nix +manual](http://nixos.org/nix/manual/). Part 3: HashDist update procedure --------------------------------- - -There are two stages to an update of the HashDist store; first updating the HashStack with the latest build instructions, then re-running the build process. To update the HashStack use git: +There are two stages to an update of the HashDist store; first updating the +HashStack with the latest build instructions, then re-running the build +process. To update the HashStack use git: $ git pull origin master -The HashDist update procedure is a repeat of the latter part of the original installation process. First, move to the HashStack directory and delete the ``default`` link: +The HashDist update procedure is a repeat of the latter part of the original +installation process. First, move to the HashStack directory and delete the +``default`` link: $ rm default -Then, for a system with M cores available for the build process, run the command: +Then, for a system with M cores available for the build process, run the +command: $ hit build -jM @@ -68,10 +88,12 @@ followed by: $ sh tweed.hashdist.post-install -HashDist deletes previous application versions and removes unused files in one command: +HashDist deletes previous application versions and removes unused files in one +command: $ hit gc -More information on HashDist is avaialable in the [HashDist manual](http://hashdist.readthedocs.io/en/latest/index.html). +More information on HashDist is avaialable in the [HashDist +manual](http://hashdist.readthedocs.io/en/latest/index.html). @@ -6,21 +6,58 @@ Copyright © 2016 Paul Garlick <[pgarlick@tourbillion-technology.com](mailto:pga notice and this notice are preserved. This file is offered as-is, without any warranty. -*TWEED* is a system for organizing engineering software. *TWEED* software applications can be moved freely, as a group, from one computer to another. All software versions can be maintained and all dependencies remain intact. *TWEED* is the **T**ransferable **W**orking **E**nvironment for **E**ngineering **D**esign. - -In the domains of computational science and computer-aided engineering it is common to develop workflows to accomplish a task or investigate a process. The workflow comprises a set of software applications and associated data. A problem with many workflows is that they are fragile. Hardware breaks, software licenses expire, data in restricted formats becomes inaccessible. It can be a major task to re-build complex, carefully constructed workflows. The purpose of *TWEED* is to make it easier to develop sustainable workflows . Using the term *sustainable* in this context is to say the workflows should be easy to move around and durable over time. - -In this model *TWEED* provides the workflow applications. The user supplies, or generates, the data. Applications are included in *TWEED* for tasks such as 3D modelling and numerical simulation. Users may choose from applications geared towards manufacturing operations, such as 2D draughting, or instead concentrate efforts on research and development using advanced optimization algorithms, as one example. Tools are included for generating grids for arbitrarily complex shapes and displaying results. - -A key feature of *TWEED* is the separation of the applications from the underlying operating system. *TWEED* may be installed on any computer running any operating system. This includes laptops and supercomputers and anything in between. All that is needed is disk space for the applications and their libraries (10 GB is recommended). - -The separation of the applications from the operating system makes the working environment transferable. This is useful for an engineer moving a workflow that has been developed on one computer to another, possibly different, computer. It is also useful in scaling a development environment, on a small system, to a production environment, on a larger system. The installation of the applications is exactly the same in each case. - -A major advantage of using *TWEED* in preference to the operating system package managers is the relative speed of upgrading packages. Whereas the upgrade cycle of the operating system packages may be in the region of 2--3 years, application upgrades in *TWEED* may be performed within days. This is especially helpful in the realm of computational science, where new methods are being introduced at a rapid pace. - -*TWEED* automates the compilation and build processes of new package versions and keeps track of dependencies. In comparison to individual installations from source, *TWEED* keeps the filesystem neat and tidy. It is also easier to maintain, having dedicated commands for upgrading packages and removing unused files. - -To make this possible *TWEED* makes use of modern package management software. Three package managers are used: +*TWEED* is a system for organizing engineering software. *TWEED* software +applications can be moved freely, as a group, from one computer to another. +All software versions can be maintained and all dependencies remain intact. +*TWEED* is the **T**ransferable **W**orking **E**nvironment for **E**ngineering +**D**esign. + +In the domains of computational science and computer-aided engineering it is +common to develop workflows to accomplish a task or investigate a process. The +workflow comprises a set of software applications and associated data. A +problem with many workflows is that they are fragile. Hardware breaks, +software licenses expire, data in restricted formats becomes inaccessible. It +can be a major task to re-build complex, carefully constructed workflows. The +purpose of *TWEED* is to make it easier to develop sustainable workflows . +Using the term *sustainable* in this context is to say the workflows should be +easy to move around and durable over time. + +In this model *TWEED* provides the workflow applications. The user supplies, +or generates, the data. Applications are included in *TWEED* for tasks such as +3D modelling and numerical simulation. Users may choose from applications +geared towards manufacturing operations, such as 2D draughting, or instead +concentrate efforts on research and development using advanced optimization +algorithms, as one example. Tools are included for generating grids for +arbitrarily complex shapes and displaying results. + +A key feature of *TWEED* is the separation of the applications from the +underlying operating system. *TWEED* may be installed on any computer running +any operating system. This includes laptops and supercomputers and anything in +between. All that is needed is disk space for the applications and their +libraries (10 GB is recommended). + +The separation of the applications from the operating system makes the working +environment transferable. This is useful for an engineer moving a workflow +that has been developed on one computer to another, possibly different, +computer. It is also useful in scaling a development environment, on a small +system, to a production environment, on a larger system. The installation of +the applications is exactly the same in each case. + +A major advantage of using *TWEED* in preference to the operating system +package managers is the relative speed of upgrading packages. Whereas the +upgrade cycle of the operating system packages may be in the region of 2--3 +years, application upgrades in *TWEED* may be performed within days. This is +especially helpful in the realm of computational science, where new methods are +being introduced at a rapid pace. + +*TWEED* automates the compilation and build processes of new package versions +and keeps track of dependencies. In comparison to individual installations +from source, *TWEED* keeps the filesystem neat and tidy. It is also easier to +maintain, having dedicated commands for upgrading packages and removing unused +files. + +To make this possible *TWEED* makes use of modern package management software. +Three package managers are used: 1. [GNU Guix](https://www.gnu.org/software/guix/) @@ -28,9 +65,15 @@ To make this possible *TWEED* makes use of modern package management software. 3. [HashDist](https://hashdist.github.io/) -A shared characteristic of these package managers is their ability to install all of the required dependencies of a given package in a unique location, alongside the main operating system. This allows for multiple versions of the same application to be installed and made available simultaneously. On a multi-user system, it is also possible for each user to individually specify their own set of preferred applications. +A shared characteristic of these package managers is their ability to install +all of the required dependencies of a given package in a unique location, +alongside the main operating system. This allows for multiple versions of the +same application to be installed and made available simultaneously. On a +multi-user system, it is also possible for each user to individually specify +their own set of preferred applications. -*TWEED* comprises an instruction set and configuration files to install the following applications: +*TWEED* comprises an instruction set and configuration files to install the +following applications: 1. [LibreCAD](http://librecad.org/cms/home.html)---2D draughting @@ -39,11 +82,20 @@ A shared characteristic of these package managers is their ability to install al 4. [FEniCS](https://fenicsproject.org/)---numerical simulation -A user may install a subset of the *TWEED* packages. It is also possible for the user to extend the list of packages, beyond the group offered by *TWEED*. For details of the installation procedure, please see the file INSTALL. +A user may install a subset of the *TWEED* packages. It is also possible for +the user to extend the list of packages, beyond the group offered by *TWEED*. +For details of the installation procedure, please see the file INSTALL. -Over time, in the development of *TWEED* itself, packages may be added to or subtracted from the included group. However, it is an objective that the size of the group will stabilize at a figure that can be reasonably maintained by a single engineer. This figure has not been fixed. Suggestions are welcome! +Over time, in the development of *TWEED* itself, packages may be added to or +subtracted from the included group. However, it is an objective that the size +of the group will stabilize at a figure that can be reasonably maintained by a +single engineer. This figure has not been fixed. Suggestions are welcome! -Infrequently, there may also be change to the list of package managers used by *TWEED*. An ideal list would contain a single package manager. This is a long-term objective. However, the list may increase or decrease in length temporarily as package management technology evolves and as the availability of applications provided by the different package managers changes. +Infrequently, there may also be change to the list of package managers used by +*TWEED*. An ideal list would contain a single package manager. This is a +long-term objective. However, the list may increase or decrease in length +temporarily as package management technology evolves and as the availability of +applications provided by the different package managers changes. Paul Garlick |