You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Added instructions for upgrading cloudera-deploy when it is checked out using git
* Added option to use the quickstart.sh script without checkout
* Added sections on checking credentials before attempting an Execution
* Various other tweaks and revisions
Signed-off-by: Daniel Chaffelson <chaffelson@gmail.com>
NOTE: Git is required if you intend to clone the software for local editing, if you just intend to Run the automation tools you may skip this step.
46
+
47
+
There are excellent instructions for installing Git on all Operating Systems on the https://git-scm.com/book/en/v2/Getting-Started-Installing-Git[Git website]
48
+
43
49
==== (Optional) Install AWS CLI
44
50
45
51
Get the latest version of the **AWS CLI**.
@@ -90,13 +96,23 @@ TIP: Put AWS CLI and CDP CLI programs in your `$PATH` to make these two programs
90
96
91
97
=== Setup
92
98
93
-
==== Clone the repository
99
+
==== Option 1: Download the Quickstart script
100
+
101
+
The `quickstart.sh` script will setup the Docker container with the software dependencies you need for deployment.
WARNING: Be careful to not modify any of the files in the project as a user of the software. The vast majority of changes are managed through configurations provided to these project files.
@@ -115,14 +131,14 @@ Run the `quickstart.sh` entrypoint script. This script will prepare and execute
115
131
116
132
[source, bash]
117
133
----
118
-
cd cloudera-deploy
119
134
chmod +x quickstart.sh
120
135
./quickstart.sh
121
136
----
122
137
123
138
==== Confirm the Quickstart environment
124
139
125
-
Confirm that you have the orange _cldr_ prompt. This is your interactive Ansible Runner environment and provides builtin access to the relevant dependencies for CDP.
140
+
Confirm that you have the orange `cldr (build)-(version) #>` prompt. +
141
+
This is your interactive Ansible Runner environment and provides builtin access to the relevant dependencies for CDP.
126
142
127
143
IMPORTANT: Do _NOT_ run the example definition until you have made the changes below.
128
144
@@ -149,10 +165,58 @@ WARNING: Please ensure you provide a valid region for the `infra_type` property.
149
165
150
166
=== Execution
151
167
168
+
==== Check your Credentials
169
+
170
+
Before running a Deployment, it is good practice to check that the credentials available to the Automation software are functioning correctly and _match the expected accounts_ - generally it is good practice to compare the user and account IDs produced in the terminal match those found in the Browser UI.
171
+
172
+
===== CDP
173
+
174
+
If you are deploying CDP Public, check your credential is available in your profile
175
+
176
+
[source, bash]
177
+
----
178
+
cdp iam get-user
179
+
----
180
+
181
+
TIP: If you do not yet have a CDP Public credential, follow the Cloudera Documentation https://docs.cloudera.com/cdp/latest/cli/topics/mc-cli-generating-an-api-access-key.html[here]
182
+
183
+
===== AWS
184
+
185
+
If you are using AWS cloud infrastructure, check your credential is available in your profile
186
+
187
+
[source, bash]
188
+
----
189
+
aws iam get-user
190
+
----
191
+
192
+
===== Azure
193
+
194
+
If you are using Azure cloud infrastructure, check you are logged into your account and your credentials are available
195
+
196
+
[source, bash]
197
+
----
198
+
az account list
199
+
----
200
+
201
+
TIP: If you cannot list your Azure accounts, consider using `az login` to refresh your credential
202
+
203
+
===== GCP
204
+
205
+
If you are using GCP cloud infrastructure, check your service account credential is being picked up.
206
+
207
+
WARNING: You need a provisioning Service Account for GCP setup in your `cloudera-deploy` user profile 'gcloud_credential_file' entry. If you do not yet have a Provisioning Service Account you can follow this process in the https://docs.cloudera.com/cdp/latest/gcp-quickstart/topics/mc-gcp-quickstart-step1.html[CDP Documentation] to generate one.
208
+
209
+
[source, bash]
210
+
----
211
+
gcloud auth list
212
+
----
213
+
152
214
==== Run the main playbook
153
215
154
216
Run the main playbook with the defaults and your configuration at the orange _cldr_ prompt.
155
217
218
+
NOTE: This will create a ' CDP sandbox', which is both a CDP Public Environment and CDP Private Base cluster using your default Cloud Infrastructure Provider credentials. Many other deployments are possible and explained elsewhere.
IMPORTANT: The total time to deploy varies from 90 to 150 minutes, depending on CDN, network connectivity, etc. Keep checking the logs; if there are no errors, the scripts are working in the background.
172
236
237
+
=== Upgrade
238
+
239
+
Cloudera-Deploy is regularly updated by the maintainers with new features and fixes. +
240
+
The `quickstart.sh` script will check for an updated Container image to use if there is currently no Container running. +
241
+
You may use the following process to trigger this behavior.
242
+
243
+
WARNING: This will close any active `cldr` sessions you may have running.
244
+
245
+
Stop the cloudera-deploy Docker Container
246
+
[source, bash]
247
+
----
248
+
docker stop cloudera-deploy
249
+
----
250
+
251
+
WARNING: If you have made local uncommitted changes to cloudera-deploy, you must resolve them before updating
252
+
253
+
In the cloudera-deploy directory, pull the latest changes with git
254
+
255
+
[source, bash]
256
+
----
257
+
git pull
258
+
----
259
+
260
+
Finally, rerun the quickstart to download the latest image.
261
+
262
+
TIP: You can stop the Docker Container and rerun the quickstart at any time to download the latest image
263
+
264
+
[source, bash]
265
+
----
266
+
./quickstart.sh
267
+
----
268
+
173
269
== Project Details
174
270
175
271
CAUTION: Don't change the project configuration without getting comfortable with the *quickstart* a few times.
176
272
177
273
NOTE: Below pages will be migrated to Github pages shortly.
178
274
179
-
Cloudera Deploy is powered by https://github.com/ansible/ansible[Ansible] and provides a standard configuration and execution model for CDP deployments and their applications. It can be run within a container or directly on a host.
275
+
Cloudera Deploy is powered by https://github.com/ansible/ansible[Ansible] and provides a standard configuration and execution model for CDP deployments and their applications. It can be run within a container, or directly on a host.
180
276
181
277
Specifically, Cloudera Deploy is an Ansible project that uses a set of playbooks, roles, and tags to construct a runlevel-like management experience for cloud and cluster deployments. It leverages several collections, both Cloudera and third-party.
182
278
183
279
=== Software Dependencies
184
280
185
-
Cloudera Deploy requires a number of host applications, services, and Python libraries for its execution. These dependencies are already packaged for ease-of-use in https://github.com/cloudera-labs/cldr-runner[Cloudera Labs Ansible-Runner], another project within Cloudera Labs.
281
+
Cloudera Deploy requires a number of host applications, services, and Python libraries for its execution. These dependencies are already packaged for ease-of-use in https://github.com/cloudera-labs/cldr-runner[Cloudera Labs Ansible-Runner], another project within Cloudera Labs, and are made readily accessible through the `quickstart.sh` script.
186
282
187
283
Alternatively, and especially if you plan on running Cloudera Deploy in your own environment, you may install the dependencies yourself.
188
284
189
285
==== Collections and Roles
190
286
191
-
Cloudera Deploy relies on a number of Ansible collections:
287
+
Cloudera Deploy relies directly on a number of Ansible collections:
@@ -229,7 +325,7 @@ The dependencies cover the full range of the automation tooling, from infrastruc
229
325
230
326
=== User Input Dependencies
231
327
232
-
Cloudera Deploy does require a small set of user-supplied information for a successful deployment. A minimum set of user inputs is defined in a _profile_ file (see the link:profile.yml[profile.yml] template for details). For example, the `profile.yml` should define your password for the Administrator account of the deployed services.
328
+
Cloudera Deploy does require a small set of user-supplied information for a successful deployment. A minimum set of user inputs is defined in a _profile_ file (see the link:profile.yml[profile.yml] template for details). For example, the `profile.yml` should define your password for the Administrator account of the deployed services, and you should set a unique `name_prefix` to avoid clashing with other deployments.
233
329
234
330
The default location for profiles is `~/.config/cloudera-deploy/profiles/`. Cloudera Deploy looks for the `default` file in this directory unless the Ansible runtime variable `profile` is set, e.g. `-e profile=my_custom_profile`. Creating additional profiles is simple, and you can use the `profile.yml` template as your starting point.
235
331
@@ -282,11 +378,11 @@ ansible-playbook <location of cloudera-deploy>/main.yml \
282
378
-e "definition_path=<absolute or relative directory to main.yml>"
283
379
----
284
380
285
-
NOTE: The location defined by `definition_path` is relative _to the location of the `main.yml` playbook_ and can also be an absolute location.
381
+
NOTE: The location defined by `definition_path` is relative _to the location of the `main.yml` playbook_, and can also be an absolute location.
286
382
287
383
=== Tags
288
384
289
-
Cloudera Deploy exposes a set of tags that allows fine-grained inclusion and exclusion of functions, in particular, a runlevel-like management process.
385
+
Cloudera Deploy exposes a set of Ansible tags that allows fine-grained inclusion and exclusion of functions, in particular, a runlevel-like management process.
290
386
291
387
.Partial List of Available Execution Tags
292
388
[cols="1,1"]
@@ -322,7 +418,9 @@ or select or skip a level or function:
322
418
ansible-playbook main.yml -e "definition_path=my_example" -t run --skip-tags infra
323
419
----
324
420
325
-
For details on the various _runlevel_-like tags for CDP Public Cloud, see the https://github.com/cloudera-labs/cloudera.exe/docs/runlevels.md[Runlevel Guide] in the `cloudera.exe` project.
421
+
WARNING: Setting a deployment to a lower runlevel, e.g. from `run` to `infra` will teardown deployed components in the higher runlevels.
422
+
423
+
For further details on the various _runlevel_-like tags for CDP Public Cloud, see the https://github.com/cloudera-labs/cloudera.exe/docs/runlevels.md[Runlevel Guide] in the `cloudera.exe` project.
0 commit comments