1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
# Tweed.README
Copyright © 2016, 2017, 2018, 2020 Paul Garlick <[pgarlick@tourbillion-technology.com](mailto:pgarlick@tourbillion-technology.com)>
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved. This file is offered as-is,
without any warranty.
## Introduction
*TWEED* is a system for organizing engineering software. *TWEED* software
applications can be moved freely, as a group, from one computer to another.
All software versions can be maintained and all dependencies remain intact.
*TWEED* is the **T**ransferable **W**orking **E**nvironment for **E**ngineering
**D**esign.
In the domains of computational science and computer-aided engineering it is
common to develop workflows to accomplish a task or investigate a process. The
workflow comprises a set of software applications and associated data. A
problem with many workflows is that they are fragile. Hardware breaks,
software licenses expire, data in restricted formats becomes inaccessible. It
can be a major task to re-build complex, carefully constructed workflows. The
purpose of *TWEED* is to make it easier to develop sustainable workflows .
Using the term *sustainable* in this context is to say the workflows should be
easy to move around and durable over time.
In this model *TWEED* provides the workflow applications. The user supplies,
or generates, the data. Applications are included in *TWEED* for tasks such as
3D modelling and numerical simulation. Users may choose from applications
geared towards manufacturing operations, such as 2D draughting, or instead
concentrate efforts on research and development using advanced optimization
algorithms, as one example. Tools are included for generating grids for
arbitrarily complex shapes and displaying results.
A key feature of *TWEED* is the separation of the applications from the
underlying operating system. *TWEED* may be installed on any computer running
any operating system. This includes laptops and supercomputers and anything in
between. All that is needed is disk space for the applications and their
libraries (10 GB is recommended).
The separation of the applications from the operating system makes the working
environment transferable. This is useful for an engineer moving a workflow
that has been developed on one computer to another, possibly different,
computer. It is also useful in scaling a development environment, on a small
system, to a production environment, on a larger system. The installation of
the applications is exactly the same in each case.
## Package Managers
A major advantage of using *TWEED* in preference to the operating system
package managers is the relative speed of upgrading packages. Whereas the
upgrade cycle of the operating system packages may be in the region of 2--3
years, application upgrades in *TWEED* may be performed within days. This is
especially helpful in the realm of computational science, where new methods are
being introduced at a rapid pace.
*TWEED* automates the compilation and build processes of new package versions
and keeps track of dependencies. In comparison to individual installations
from source, *TWEED* keeps the filesystem neat and tidy. It is also easier to
maintain, having dedicated commands for upgrading packages and removing unused
files.
To make this possible *TWEED* makes use of modern package management software.
Three package managers are used:
1. [GNU Guix](https://www.gnu.org/software/guix/)
2. [Nix](http://nixos.org/nix/)
3. [HashDist](https://hashdist.github.io/)
A shared characteristic of these package managers is their ability to install
all of the required dependencies of a given package in a unique location,
alongside the main operating system. This allows for multiple versions of the
same application to be installed and made available simultaneously. On a
multi-user system, it is also possible for each user to individually specify
their own set of preferred applications.
## Applications
*TWEED* comprises an instruction set and configuration files to install the
following applications:
1. [LibreCAD](http://librecad.org/cms/home.html)---2D draughting
2. [FreeCAD](http://www.freecadweb.org/)---3D modelling
3. [Gmsh](http://gmsh.info/)---grid generation
4. [FEniCS](https://fenicsproject.org/)---numerical simulation (finite element)
5. [OpenFOAM](https://openfoam.org/)---numerical simulation (finite volume)
6. [ParaView](https://www.paraview.org/)---post processing
## Directory Structure
The directory structure for a typical *TWEED* installation is shown in
the diagram below. The configuration files are stored in the *tweed*
directory, located in the user's home directory. The three package
managers use the directories $HOME/.hashdist, /gnu and /nix
respectively to store the application files. GNU Guix uses a
sub-directory in /etc to store public keys of authorised servers and a
sub-directory in /var to store log files and database files.
~~~
/ (root)
├── OS
├── etc
│ └── guix
├── var
│ ├── guix
│ └── log
│ └── guix
│
├── $HOME
│ ├── .hashdist
│ └── tweed
│ ├── hashdist
│ └── hashstack
│
├── gnu
└── nix
~~~
A user may install a subset of the *TWEED* packages. It is also possible for
the user to extend the list of packages, beyond the group offered by *TWEED*.
For details of the installation procedure, please see the file INSTALL.
## Data Formats
The applications in *TWEED* make use of and generate data. Each application
defines its own internal data structure, optimised for that application in
particular. When there is a need to transfer data from one application to
another it is necessary to use a data format that is recognised by both. There
are a number of industry standard data formats for this purpose. These are
well-established and widely used. In addition, there are specific data formats
used by particular applications only. *TWEED* applications are able to make
use of both types of data format. Communication between the applications is
possible in a number of ways. A configuration of applications and exchange
formats that demonstrates the principle is shown in the diagram below:
This configuration forms a template for a typical engineering workflow. It is
suitable for a wide range of design and development projects. The *Gmsh*
application is central in this configuration. It provides the pre- and
post-processing facilities for computer simulations. *Gmsh* is capable of
generating the geometry for simple meshes itself. For more complex cases the
geometry can be defined in *FreeCAD* and passed on to *Gmsh*. Finite element
models, using the meshes from *Gmsh*, can be solved with *FEniCS*. Finite
volume models can be solved with *OpenFOAM*. Result files can be passed back
to *Gmsh* or on to *ParaView* for post-processing.
In this configuration there are three standard file formats:
1. [DXF](https://www.autodesk.com/techpubs/autocad/acadr14/dxf/ascii_dxf_file_format_al_u05_b.htm) - 2D CAD data
2. [STEP](http://www.steptools.com/stds/step/) - 3D CAD data
3. [XDMF](http://www.xdmf.org/index.php/XDMF_Model_and_Format) - mesh and result data
and two specific formats:
1. [MSH](http://gmsh.info/doc/texinfo/gmsh.html#File-formats) - *Gmsh* mesh and result data
2. [FOAM](https://cfd.direct/openfoam/user-guide/basic-file-format/) - *OpenFOAM* mesh and result data
All of the applications in *TWEED* can read from and write to at least one
standard data format. For example, *FEniCS* can write out result files in
*xdmf* format. These files can subsequently be read in by *ParaView*.
However, the best method for communication with the *Gmsh* application is to
use the *Gmsh*-specific *msh* format. To do this, input/output (i/o) scripts
are used to convert the data to and from the numerical solver formats.
In the case of *FEniCS*, the multi-format utility
[meshio](https://github.com/nschloe/meshio) can be used to convert between
*xdmf* and *msh* formats. For *OpenFOAM*, *gmshToFoam* is available, supplied
as one of the mesh conversion utilities. It is also possible to convert
*OpenFOAM* result data to *Gmsh msh* format. An example script is
*foam2gmsh.py,* available from the [ONELAB
wiki](http://onelab.info/wiki/ONELAB).
## Future
Over time, in the development of *TWEED* itself, packages may be added to or
subtracted from the included group. However, it is an objective that the size
of the group will stabilize at a figure that can be reasonably maintained by a
single engineer. This figure has not been fixed. Suggestions are welcome!
Infrequently, there may also be change to the list of package managers used by
*TWEED*. An ideal list would contain a single package manager. This is a
long-term objective. However, the list may increase or decrease in length
temporarily as package management technology evolves and as the availability of
applications provided by the different package managers changes.
Paul Garlick
April 2018
|
