Skip to content

Commit 112b3db

Browse files
authored
Merge pull request #9 from bedroge/master
Documentation and playbook improvements
2 parents 2d8d7b9 + 72c5a1c commit 112b3db

14 files changed

+259
-37
lines changed

.gitmodules

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
[submodule "roles/galaxyproject.cvmfs"]
22
path = roles/cvmfs
33
url = https://github.com/galaxyproject/ansible-cvmfs
4+
[submodule "roles/geerlingguy.repo-epel"]
5+
path = roles/geerlingguy.repo-epel
6+
url = https://github.com/geerlingguy/ansible-role-repo-epel

README.md

Lines changed: 150 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,30 +20,171 @@ The CVMFS layer of the EESSI project consists of the usual CVMFS infrastructure:
2020

2121
## Installation and configuration
2222

23-
For the installation of all components we make use of the Ansible files provided by the Galaxy project:
24-
https://github.com/galaxyproject/ansible-cvmfs
25-
This repository is added as a submodule inside the roles directory, so make sure to use the --recursive options when cloning this repository:
23+
### Prerequisites
24+
25+
The main prerequisite is Ansible (https://github.com/ansible/ansible),
26+
which can be easily installed via the package manager of most Linux distributions or via `pip install`.
27+
For more details, see the Ansible installation guide: https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html.
28+
29+
For the installation of all components we make use of two Ansible roles: the CVMFS installation role provided by the Galaxy project (see
30+
https://github.com/galaxyproject/ansible-cvmfs), and a role for adding the EPEL repository (https://github.com/geerlingguy/ansible-role-repo-epel).
31+
Both repositories are added as a submodule inside the `roles` directory, so make sure to use the `--recursive` option when cloning this repository:
2632
```
2733
git clone --recursive git@github.com:EESSI/cvmfs-layer.git
2834
```
29-
Alternatively, clone this repository first, and init and update the required submodule later:
35+
Alternatively, clone this repository first, and init and update the required submodules later:
3036
```
3137
git clone git@github.com:EESSI/cvmfs-layer.git
32-
cd cvmfs-layer/roles/cvmfs/
38+
cd cvmfs-layer
3339
git submodule init
3440
git submodule update
3541
```
3642
For more information about (working with) submodules, see:
3743
https://git-scm.com/book/en/v2/Git-Tools-Submodules
3844

39-
The EESSI specific settings can be found in group_vars/all.yml, and in templates we added a template for a Squid configation for the local proxy servers; this file is not included in the Galaxy repository.
45+
### Configuration
46+
47+
The EESSI specific settings can be found in `group_vars/all.yml`, and in `templates` we added our own templates
48+
of Squid configurations for the Stratum 1 and local proxy servers.
49+
For all playbooks you will also need to have an appropriate Ansible `hosts` file;
50+
see the supplied `hosts.example` for the structure and host groups that you need for these playbooks.
51+
52+
## Running the playbooks
4053

41-
In order to run one of these playbooks, you will have to use a hosts file that includes at least the group of nodes for which you are running a playbook. An example of the file that shows the correct structure can be found in hosts.example.
42-
The playbooks can then be run as follows:
54+
In general, all the playbooks can be run like this:
4355
```
4456
ansible-playbook -i hosts -b <name of playbook>.yml
4557
```
46-
where -b means become, i.e. run with sudo. If this requires a password, include -K:
58+
where `-i` allows you to specify the path to your hosts file, and `-b` means "become", i.e. run with `sudo`.
59+
If this requires a password, include `-K`, which will ask for the `sudo` password when running the playbook:
4760
```
4861
ansible-playbook -i hosts -b -K <name of playbook>.yml
4962
```
63+
64+
Before you run any of the commands below, make sure that you updated the file `group_vars/all.yml`
65+
and include the new/extra URLs of any server you want to change/add (e.g. add your Stratum 1).
66+
67+
68+
### Stratum 0
69+
First install the Stratum 0 server:
70+
```
71+
ansible-playbook -i hosts -b -K stratum0.yml
72+
```
73+
74+
Then install the files for the configuration repository:
75+
```
76+
ansible-playbook -i hosts -b -K stratum0-deploy-cvmfs-config.yml
77+
```
78+
79+
Note that there can be only one Stratum 0, so you should only run this playbook
80+
for testing purposes or in case we need to move or redeploy the current Stratum 0 server.
81+
82+
### Stratum 1
83+
Installing a Stratum 1 requires a GEO API license key, which will be used to find
84+
the (geographically) closest Stratum 1 server for your client and proxies.
85+
More information on how to (freely) obtain this key is available in the CVMFS documentation:
86+
https://cvmfs.readthedocs.io/en/stable/cpt-replica.html#geo-api-setup .
87+
88+
You can put your license key in `group_vars/all.yml`, or add a section in your `hosts` file:
89+
```yaml
90+
[cvmfsstratum1servers:vars]
91+
cvmfs_geo_license_key=XXXXX
92+
```
93+
94+
Furthermore, the Stratum 1 runs a Squid server. The template configuration file can be found at
95+
`templates/eessi_stratum1_squid.conf.j2`.
96+
If you want to customize it, for instance for limiting the access to the Stratum 1,
97+
you can make your own version of this template file and point to it by editing the playbook or
98+
adding the following to `group_vars/all.yml` or the section in your `hosts` file:
99+
```yaml
100+
cvmfs_squid_conf_src=/path/to/your_stratum1_squid.conf.j2
101+
```
102+
Install the Stratum 1 using:
103+
```
104+
ansible-playbook -i hosts -b -K stratum1.yml
105+
```
106+
This will automatically make replicas of all the repositories defined in `group_vars/all.yml`.
107+
108+
### Local proxies
109+
The local proxies also need a Squid configuration file; the default can be found in
110+
`templates/localproxy_squid.conf.j2`.
111+
112+
You have to define the lists of IP addresses / ranges (using CIDR notation) that are allowed to use the proxy using the variable `cvmfs_localproxy_allowed_clients`.
113+
You can put this, for instance, in your hosts file. See `hosts.example` for more details.
114+
115+
If you want to customize the Squid configuration more, you can also make your own file, and point to it using `cvmfs_squid_conf_src` (see the Stratum 1 section).
116+
117+
Do keep in mind that you should never accept proxy request from everywhere to everywhere!
118+
Besides having a Squid configuration with the right ACLs, it is recommended to also have a firewall that limits access to your proxy.
119+
120+
Deploy your proxies using:
121+
```
122+
ansible-playbook -i hosts -b -K localproxy.yml
123+
```
124+
125+
### Clients
126+
Make sure that your hosts file contains the list of hosts where the CVMFS client should be installed.
127+
Furthermore, you can add a vars section for the clients that contains the list of (local) proxy servers
128+
that your clients should use:
129+
```yaml
130+
[cvmfsclients:vars]
131+
cvmfs_http_proxies=["your-local.proxy:3128"]
132+
```
133+
If you just want to roll out one client without a proxy, you can leave this out.
134+
Finally, run the playbook:
135+
```
136+
ansible-playbook -i hosts -b -K client.yml
137+
```
138+
139+
## Verification and usage
140+
141+
### Client
142+
143+
Once the client has been installed, you should be able to access all repositories under /cvmfs. They might not immediately show up in that directory before you have actually used them, so you might first have to run ls, e.g.:
144+
```
145+
ls /cvmfs/cvmfs-config.eessi-hpc.org
146+
```
147+
148+
On the client machines you can use the `cvmfs_config` tool for different operations. For instance, you can verify the file system by running:
149+
```
150+
$ sudo cvmfs_config probe cvmfs-config.eessi-hpc.org
151+
Probing /cvmfs/cvmfs-config.eessi-hpc.org... OK
152+
```
153+
154+
Checking for misconfigurations can be done with:
155+
```
156+
sudo cvmfs_config chksetup
157+
```
158+
159+
In case of unclear issues, you can enable the debug mode and log to a file by setting the following environment variable:
160+
```
161+
CVMFS_DEBUGFILE=/some/path/to/cvmfs.log
162+
```
163+
164+
### Proxy / Stratum 1
165+
166+
In order to test your local proxy and/or Stratum 1, even without a client installed, you can use curl:
167+
```
168+
curl --proxy http://url-to-your-proxy:3128 --head http://url-to-your-stratum1/cvmfs/cvmfs-config.eessi-hpc.org/.cvmfspublished
169+
```
170+
This should return:
171+
```
172+
HTTP/1.1 200 OK
173+
...
174+
X-Cache: MISS from url-to-your-proxy
175+
```
176+
The second time you run it, you should get a cache hit:
177+
```
178+
X-Cache: HIT from url-to-your-proxy
179+
```
180+
181+
### Using the CVMFS infrastructure
182+
183+
When the infrastructure seems to work, you can try publishing some new files. This can be done by starting a transaction on the Stratum 0, adding some files, and publishing the transaction:
184+
```
185+
sudo cvmfs_server transaction pilot.eessi-hpc.org
186+
mkdir /cvmfs/pilot.eessi-hpc.org/testdir
187+
touch /cvmfs/pilot.eessi-hpc.org/testdir/testfile
188+
sudo cvmfs_server publish pilot.eessi-hpc.org
189+
```
190+
It might take a few minutes, but then the new file should show up at the clients.

client.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
- name: CVMFS
1+
# Install CVMFS clients.
2+
---
3+
- name: CVMFS clients
24
hosts: cvmfsclients
35
vars:
46
eessi_cvmfs_repos_enabled: config-repo

group_vars/all.yml

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
# https://github.com/EESSI/cvmfs-layer/issues/2
88
#cvmfs_geo_license_key:
99

10-
# Automatically configure EESSI CVMFS repos
10+
# Automatically configure EESSI CVMFS repos.
1111
eessi_cvmfs_repos_enabled: config-repo
1212

1313
# Email address for the project, which will be put in the contact file on the config repo.
@@ -34,10 +34,17 @@ eessi_cvmfs_config_repo:
3434
repository: cvmfs-config.eessi-hpc.org
3535
stratum0: cvmfs-s0.eessi-hpc.org
3636
owner: "{{ cvmfs_repo_owner | default('root') }}"
37+
key_dir: /etc/cvmfs/keys/eessi-hpc.org
3738
server_options: []
3839
client_options: []
3940

40-
# Defaults for eessi-hpc.org repos
41+
#
42+
# Defaults for eessi-hpc.org repos.
43+
#
44+
45+
# Public keys for the repositories, which you can find on the Stratum 0 at:
46+
# /etc/cvmfs/keys/*.pub
47+
# Note: you first have to run the stratum0.yml playbook once to generate the repositories and keys.
4148
eessi_cvmfs_keys:
4249
- path: /etc/cvmfs/keys/eessi-hpc.org/pilot.eessi-hpc.org.pub
4350
key: |
@@ -51,12 +58,14 @@ eessi_cvmfs_keys:
5158
FQIDAQAB
5259
-----END PUBLIC KEY-----
5360
61+
# URLs for all the Stratum 1 servers.
5462
eessi_cvmfs_server_urls:
5563
- domain: eessi-hpc.org
5664
use_geoapi: true
5765
urls:
5866
- "http://cvmfs-s1-rug.eessi-hpc.org/cvmfs/@fqrn@"
5967

68+
# Configuration of all the repositories.
6069
eessi_cvmfs_repositories:
6170
- repository: pilot.eessi-hpc.org
6271
stratum0: cvmfs-s0.eessi-hpc.org

hosts.example

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,21 @@ your-stratum1.org
88
your-proxy-1
99
your-proxy-2
1010

11+
[cvmfslocalproxies:vars]
12+
# List of clients allowed to access the proxy.
13+
# Add individual IPs and/or use CIDR notation.
14+
cvmfs_localproxy_allowed_clients = ["192.168.0.0/12", "10.0.0.15"]
15+
1116
[cvmfsclients]
1217
your-client-1
1318
your-client-2
1419

1520
[cvmfsclients:vars]
16-
cvmfs_http_proxies=["your-proxy-1:3128", "your-proxy-2:3128"]
17-
18-
21+
# List of all http proxies that should be configured for the clients.
22+
# Remove or comment the line if you do not want to use a proxy; in this
23+
# case it will be set to "DIRECT" in the client configuration.
24+
cvmfs_http_proxies = ["your-proxy-1:3128", "your-proxy-2:3128"]
25+
# The following one-liner can be used to automatically add all the hosts
26+
# defined in the cvmfslocalproxies group to cvmfs_http_proxes,
27+
# using port number 3128.
28+
# cvmfs_http_proxies = "{{ groups.cvmfslocalproxies | map('regex_replace', '^(.*)$', '\\1:3128') | list }}"

localproxy.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
- name: CVMFS
1+
# Install CVMFS local proxy (Squid) server(s).
2+
---
3+
- name: CVMFS local proxies.
24
hosts: cvmfslocalproxies
5+
vars:
6+
cvmfs_squid_conf_src: eessi_localproxy_squid.conf.j2
37
roles:
48
- cvmfs

roles/README.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,15 @@
1-
This roles directory contains a submodule that points to the repository of the Galaxy Ansible role for installing/configuring CVMFS, which can be found at:
1+
This roles directory contains submodules that point to the repositories of Ansible roles on which the EESSI playbooks depend.
2+
3+
## cvmfs
4+
5+
The Galaxy Ansible role for installing/configuring CVMFS, which can be found at:
26
https://github.com/galaxyproject/ansible-cvmfs
37

48
We renamed the directory to "cvmfs", so we can use "cvmfs" as the name of this role in our Ansible playbooks.
9+
10+
## geerlingguy.repo-epel
11+
12+
Ansible role for adding the EPEL repository to RHEL/CentOS systems. The source can be found at:
13+
https://github.com/geerlingguy/ansible-role-repo-epel
14+
15+
We renamed the directory to "geerlingguy.repo-epel", as this is also the name used in the Galaxy CVMFS examples.

roles/geerlingguy.repo-epel

Submodule geerlingguy.repo-epel added at 15c36e4

stratum0-deploy-cvmfs-config.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# Note: first deploy your Stratum 0 and make sure that group_vars/all.yml contains the right configuration,
33
# including all the keys of the repositories that have to be configured.
44
---
5-
- name: CVMFS
5+
- name: CVMFS Stratum 0 deploy config repository
66
hosts: cvmfsstratum0servers
77
tasks:
88
- name: Start the CVMFS transaction, create directories, and copy files

stratum0.yml

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
1-
- name: CVMFS
1+
# Install a CVMFS Stratum 0 server.
2+
---
3+
- name: CVMFS Stratum 0
24
hosts: cvmfsstratum0servers
35
vars:
4-
cvmfs_numfiles: 4096
5-
cvmfs_server_urls:
6-
- domain: eessi-hpc.org
7-
urls:
8-
- "http://cvmfs-s0.eessi-hpc.org/cvmfs/@fqrn@"
9-
cvmfs_repositories: "{{ eessi_cvmfs_repositories }}"
6+
cvmfs_repositories: "{{ eessi_cvmfs_repositories + [eessi_cvmfs_config_repo.repository] }}"
107
roles:
8+
- geerlingguy.repo-epel
119
- cvmfs

0 commit comments

Comments
 (0)