Skip to content

Commit 0dba6e9

Browse files
authored
Readme Updates (#32)
* Added instructions for upgrading cloudera-deploy when it is checked out using git * Added option to use the quickstart.sh script without checkout * Added sections on checking credentials before attempting an Execution * Various other tweaks and revisions Signed-off-by: Daniel Chaffelson <chaffelson@gmail.com>
1 parent 15212f8 commit 0dba6e9

File tree

1 file changed

+109
-11
lines changed

1 file changed

+109
-11
lines changed

readme.adoc

Lines changed: 109 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,12 @@ Get the latest version of **Docker**.
4040
* https://docs.docker.com/docker-for-mac/install/[For Macs]
4141
* Linux users, use your package manager.
4242

43+
==== (Optional) Install Git
44+
45+
NOTE: Git is required if you intend to clone the software for local editing, if you just intend to Run the automation tools you may skip this step.
46+
47+
There are excellent instructions for installing Git on all Operating Systems on the https://git-scm.com/book/en/v2/Getting-Started-Installing-Git[Git website]
48+
4349
==== (Optional) Install AWS CLI
4450

4551
Get the latest version of the **AWS CLI**.
@@ -90,13 +96,23 @@ TIP: Put AWS CLI and CDP CLI programs in your `$PATH` to make these two programs
9096

9197
=== Setup
9298

93-
==== Clone the repository
99+
==== Option 1: Download the Quickstart script
100+
101+
The `quickstart.sh` script will setup the Docker container with the software dependencies you need for deployment.
102+
103+
[source, bash]
104+
----
105+
curl https://raw.githubusercontent.com/cloudera-labs/cloudera-deploy/main/quickstart.sh -o quickstart.sh
106+
----
107+
108+
==== Option 2: Clone the repository
94109

95-
Clone this, i.e. the `cloudera-deploy`, repository.
110+
Clone this, i.e. the `cloudera-deploy`, repository, which contains the `quickstart.sh` script.
96111

97112
[source, bash]
98113
----
99114
git clone https://github.com/cloudera-labs/cloudera-deploy.git
115+
cd cloudera-deploy
100116
----
101117

102118
WARNING: Be careful to not modify any of the files in the project as a user of the software. The vast majority of changes are managed through configurations provided to these project files.
@@ -115,14 +131,14 @@ Run the `quickstart.sh` entrypoint script. This script will prepare and execute
115131

116132
[source, bash]
117133
----
118-
cd cloudera-deploy
119134
chmod +x quickstart.sh
120135
./quickstart.sh
121136
----
122137

123138
==== Confirm the Quickstart environment
124139

125-
Confirm that you have the orange _cldr_ prompt. This is your interactive Ansible Runner environment and provides builtin access to the relevant dependencies for CDP.
140+
Confirm that you have the orange `cldr (build)-(version) #>` prompt. +
141+
This is your interactive Ansible Runner environment and provides builtin access to the relevant dependencies for CDP.
126142

127143
IMPORTANT: Do _NOT_ run the example definition until you have made the changes below.
128144

@@ -149,10 +165,58 @@ WARNING: Please ensure you provide a valid region for the `infra_type` property.
149165

150166
=== Execution
151167

168+
==== Check your Credentials
169+
170+
Before running a Deployment, it is good practice to check that the credentials available to the Automation software are functioning correctly and _match the expected accounts_ - generally it is good practice to compare the user and account IDs produced in the terminal match those found in the Browser UI.
171+
172+
===== CDP
173+
174+
If you are deploying CDP Public, check your credential is available in your profile
175+
176+
[source, bash]
177+
----
178+
cdp iam get-user
179+
----
180+
181+
TIP: If you do not yet have a CDP Public credential, follow the Cloudera Documentation https://docs.cloudera.com/cdp/latest/cli/topics/mc-cli-generating-an-api-access-key.html[here]
182+
183+
===== AWS
184+
185+
If you are using AWS cloud infrastructure, check your credential is available in your profile
186+
187+
[source, bash]
188+
----
189+
aws iam get-user
190+
----
191+
192+
===== Azure
193+
194+
If you are using Azure cloud infrastructure, check you are logged into your account and your credentials are available
195+
196+
[source, bash]
197+
----
198+
az account list
199+
----
200+
201+
TIP: If you cannot list your Azure accounts, consider using `az login` to refresh your credential
202+
203+
===== GCP
204+
205+
If you are using GCP cloud infrastructure, check your service account credential is being picked up.
206+
207+
WARNING: You need a provisioning Service Account for GCP setup in your `cloudera-deploy` user profile 'gcloud_credential_file' entry. If you do not yet have a Provisioning Service Account you can follow this process in the https://docs.cloudera.com/cdp/latest/gcp-quickstart/topics/mc-gcp-quickstart-step1.html[CDP Documentation] to generate one.
208+
209+
[source, bash]
210+
----
211+
gcloud auth list
212+
----
213+
152214
==== Run the main playbook
153215

154216
Run the main playbook with the defaults and your configuration at the orange _cldr_ prompt.
155217

218+
NOTE: This will create a ' CDP sandbox', which is both a CDP Public Environment and CDP Private Base cluster using your default Cloud Infrastructure Provider credentials. Many other deployments are possible and explained elsewhere.
219+
156220
[source, bash]
157221
----
158222
ansible-playbook /opt/cloudera-deploy/main.yml -e "definition_path=examples/sandbox" \
@@ -170,25 +234,57 @@ tail -100f $HOME/.config/cloudera-deploy/log/latest-2021-05-08_150448
170234

171235
IMPORTANT: The total time to deploy varies from 90 to 150 minutes, depending on CDN, network connectivity, etc. Keep checking the logs; if there are no errors, the scripts are working in the background.
172236

237+
=== Upgrade
238+
239+
Cloudera-Deploy is regularly updated by the maintainers with new features and fixes. +
240+
The `quickstart.sh` script will check for an updated Container image to use if there is currently no Container running. +
241+
You may use the following process to trigger this behavior.
242+
243+
WARNING: This will close any active `cldr` sessions you may have running.
244+
245+
Stop the cloudera-deploy Docker Container
246+
[source, bash]
247+
----
248+
docker stop cloudera-deploy
249+
----
250+
251+
WARNING: If you have made local uncommitted changes to cloudera-deploy, you must resolve them before updating
252+
253+
In the cloudera-deploy directory, pull the latest changes with git
254+
255+
[source, bash]
256+
----
257+
git pull
258+
----
259+
260+
Finally, rerun the quickstart to download the latest image.
261+
262+
TIP: You can stop the Docker Container and rerun the quickstart at any time to download the latest image
263+
264+
[source, bash]
265+
----
266+
./quickstart.sh
267+
----
268+
173269
== Project Details
174270

175271
CAUTION: Don't change the project configuration without getting comfortable with the *quickstart* a few times.
176272

177273
NOTE: Below pages will be migrated to Github pages shortly.
178274

179-
Cloudera Deploy is powered by https://github.com/ansible/ansible[Ansible] and provides a standard configuration and execution model for CDP deployments and their applications. It can be run within a container or directly on a host.
275+
Cloudera Deploy is powered by https://github.com/ansible/ansible[Ansible] and provides a standard configuration and execution model for CDP deployments and their applications. It can be run within a container, or directly on a host.
180276

181277
Specifically, Cloudera Deploy is an Ansible project that uses a set of playbooks, roles, and tags to construct a runlevel-like management experience for cloud and cluster deployments. It leverages several collections, both Cloudera and third-party.
182278

183279
=== Software Dependencies
184280

185-
Cloudera Deploy requires a number of host applications, services, and Python libraries for its execution. These dependencies are already packaged for ease-of-use in https://github.com/cloudera-labs/cldr-runner[Cloudera Labs Ansible-Runner], another project within Cloudera Labs.
281+
Cloudera Deploy requires a number of host applications, services, and Python libraries for its execution. These dependencies are already packaged for ease-of-use in https://github.com/cloudera-labs/cldr-runner[Cloudera Labs Ansible-Runner], another project within Cloudera Labs, and are made readily accessible through the `quickstart.sh` script.
186282

187283
Alternatively, and especially if you plan on running Cloudera Deploy in your own environment, you may install the dependencies yourself.
188284

189285
==== Collections and Roles
190286

191-
Cloudera Deploy relies on a number of Ansible collections:
287+
Cloudera Deploy relies directly on a number of Ansible collections:
192288

193289
- https://github.com/cloudera-labs/cloudera.exe[`cloudera.exe`]
194290
- https://github.com/cloudera-labs/cloudera.cluster[`cloudera.cluster`]
@@ -229,7 +325,7 @@ The dependencies cover the full range of the automation tooling, from infrastruc
229325

230326
=== User Input Dependencies
231327

232-
Cloudera Deploy does require a small set of user-supplied information for a successful deployment. A minimum set of user inputs is defined in a _profile_ file (see the link:profile.yml[profile.yml] template for details). For example, the `profile.yml` should define your password for the Administrator account of the deployed services.
328+
Cloudera Deploy does require a small set of user-supplied information for a successful deployment. A minimum set of user inputs is defined in a _profile_ file (see the link:profile.yml[profile.yml] template for details). For example, the `profile.yml` should define your password for the Administrator account of the deployed services, and you should set a unique `name_prefix` to avoid clashing with other deployments.
233329

234330
The default location for profiles is `~/.config/cloudera-deploy/profiles/`. Cloudera Deploy looks for the `default` file in this directory unless the Ansible runtime variable `profile` is set, e.g. `-e profile=my_custom_profile`. Creating additional profiles is simple, and you can use the `profile.yml` template as your starting point.
235331

@@ -282,11 +378,11 @@ ansible-playbook <location of cloudera-deploy>/main.yml \
282378
-e "definition_path=<absolute or relative directory to main.yml>"
283379
----
284380

285-
NOTE: The location defined by `definition_path` is relative _to the location of the `main.yml` playbook_ and can also be an absolute location.
381+
NOTE: The location defined by `definition_path` is relative _to the location of the `main.yml` playbook_, and can also be an absolute location.
286382

287383
=== Tags
288384

289-
Cloudera Deploy exposes a set of tags that allows fine-grained inclusion and exclusion of functions, in particular, a runlevel-like management process.
385+
Cloudera Deploy exposes a set of Ansible tags that allows fine-grained inclusion and exclusion of functions, in particular, a runlevel-like management process.
290386

291387
.Partial List of Available Execution Tags
292388
[cols="1,1"]
@@ -322,7 +418,9 @@ or select or skip a level or function:
322418
ansible-playbook main.yml -e "definition_path=my_example" -t run --skip-tags infra
323419
----
324420

325-
For details on the various _runlevel_-like tags for CDP Public Cloud, see the https://github.com/cloudera-labs/cloudera.exe/docs/runlevels.md[Runlevel Guide] in the `cloudera.exe` project.
421+
WARNING: Setting a deployment to a lower runlevel, e.g. from `run` to `infra` will teardown deployed components in the higher runlevels.
422+
423+
For further details on the various _runlevel_-like tags for CDP Public Cloud, see the https://github.com/cloudera-labs/cloudera.exe/docs/runlevels.md[Runlevel Guide] in the `cloudera.exe` project.
326424

327425
== Definitions
328426

0 commit comments

Comments
 (0)