Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions extend/component/deployment/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,8 +299,7 @@ allows you to obtain the repository for a component and push credentials to that
[running components](/extend/component/running/#running-a-component), for example, how to obtain the AWS registry credentials.
If you want to get even more low level, you can use the [Developer Portal API](https://kebooladeveloperportal.docs.apiary.io/#) directly.
It also allows you to [generate credentials for a service account](https://kebooladeveloperportal.docs.apiary.io/#reference/0/vendor/create-service-account)
programmatically. Apart from our AWS ECR registry, we also support running images stored in [Quay.io](https://quay.io/repository/)
and [Docker Hub](https://hub.docker.com/) registries.
programmatically. We use our AWS ECR registry for hosting all component images.

## Test Live Configurations
Testing your component can be simply added as part of the script in `.travis.yml` file. See an example in
Expand Down
7 changes: 3 additions & 4 deletions extend/component/docker-tutorial/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,9 @@ Docker Images are created by executing the instructions written in a **Dockerfil
file consisting mostly of shell commands which must be executed to prepare the application for running.
Docker images can be based on other images. So if
you need minor modification to a system, you do not have to build the whole thing from scratch. If you want Images to be
reused, *push* your Dockerfile to a **Docker registry**. The registries ([Dockerhub](https://hub.docker.com/),
[Quay](https://quay.io/)) will build the image; anyone interested in using it can download it.
[AWS ECR](https://aws.amazon.com/ecr/) is a private repository and has no build triggers. You need to push the images manually or
using a [deploy script](/extend/component/deployment/) in your CI pipeline.
reused, *push* your Dockerfile to a **Docker registry**. For Keboola components, we use
[AWS ECR](https://aws.amazon.com/ecr/). You need to push the images using a
[deploy script](/extend/component/deployment/) in your CI pipeline.

Docker image names are based on the following scheme: `registry-name/account-name/image-name:tag` where _registry-name_
and _account-name_ can sometimes be omitted. For example, you can refer to a Docker _hello-world_ image as: `hello-world`
Expand Down
9 changes: 3 additions & 6 deletions extend/component/docker-tutorial/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,8 @@ redirect_from:

An important part of the Docker ecosystem is a **Docker registry**. It acts as a folder of images, taking
care of their storing and building.
[Docker Hub](https://hub.docker.com/) is the official Docker registry.

For reliability reasons, we strongly recommend to use the [Amazon AWS ECR](https://aws.amazon.com/ecr/)
[provisioned by the **Keboola Developer Portal**](/extend/component/deployment/).
We also support Docker Hub and [Quay](https://quay.io/), both public and private repositories.
For reliability reasons, we use the [Amazon AWS ECR](https://aws.amazon.com/ecr/)
[provisioned by the **Keboola Developer Portal**](/extend/component/deployment/). This is the only supported Docker registry for Keboola components.

## Working with Registry
In order to run an image, **pull** (`docker pull`) the image to your machine. The `docker run`
Expand Down Expand Up @@ -88,4 +85,4 @@ trigger a new build of the Docker image. Also note that the image automatically
or branch name. So, when you push a commit to the `master` branch, you will get an image with a tag `master` (which
will move away from any older image builds). When creating a `1.0.0` tag, you will get an image with a `1.0.0` tag.

When using images in Keboola, we **highly recommend to use our [ECR repository](/extend/component/deployment/)**.
When using images in Keboola, you must use our [ECR repository](/extend/component/deployment/).
122 changes: 61 additions & 61 deletions extend/component/docker-tutorial/setup.md
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Špatné line breaks, proto ten +-61.

Original file line number Diff line number Diff line change
@@ -1,61 +1,61 @@
---
title: Installation and Running
permalink: /extend/component/docker-tutorial/setup/
redirect_from:
- /extend/docker/tutorial/setup/
---
* TOC
{:toc}
To work with Docker, you need to have it installed. If you do not have a computer with a Docker machine, you can obtain a
cheap [hosted server](https://marketplace.digitalocean.com/apps/docker). You can also
run everything locally. To install a Docker machine on Win/Mac,
use [Docker Community Edition](https://hub.docker.com/search?type=edition&offering=community). For
other systems, see the [documentation](https://docs.docker.com/install/).
## Getting Started
To test that everything is running correctly, start with an example
from the Docker [documentation](https://docs.docker.com/get-started/).
Run the following commands on the command line:
docker run hello-world
or
docker run docker.io/library/hello-world:latest
If this works, use any image published in any Docker
registry, e.g. [Docker Hub](https://hub.docker.com/), [Quay](https://quay.io/) or [AWS ECR](https://aws.amazon.com/ecr/).
Note that in some configurations, you may need to use `sudo` to run `docker`. If you run into problems, consult the
official troubleshooting ([Windows](https://docs.docker.com/docker-for-windows/troubleshoot/), [Mac](https://docs.docker.com/docker-for-mac/troubleshoot/)).
## Generally Useful Commands
- [`docker run`](https://docs.docker.com/engine/reference/run/): Run an
image (create a container and run the command in `ENTRYPOINT` or `CMD` section of **Dockerfile**).
- [`docker build`](https://docs.docker.com/engine/reference/commandline/build/): Build
an image (execute instructions in **Dockerfile** and create a runnable image).
- [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/): Pull
a newer version of an image (force update of the cached copy).
## Sharing Files
Sharing files between the host OS and the container is done using the `--volume` parameter of the `docker run` command:
docker run --volume=/hostPath/:/containerPath/ imageName
Do not use any spaces around `:`. The details of file sharing are somewhat
[dependent on the host OS](https://docs.docker.com/storage/volumes/) you are using.
There is a very
useful [guide for Rocker image](https://github.com/rocker-org/rocker/wiki/Sharing-files-with-host-machine), which
describes all the sharing options in great detail.
On Linux systems, where Docker runs natively, there is really not much to think about. The only things that can bite
you are file and directory permissions. Keep in mind that files created within the running Docker container on
a mounted host volume will be created with the permissions of users of that container (not the host OS).
On Mac OS, follow the [official example](https://docs.docker.com/docker-for-mac/osxfs/).
The limitation is that your host path must be in a shared directory (e.g., `/Users/`).
On Windows, follow the [official example](https://docs.docker.com/docker-for-windows/#shared-drives).
The limitation is that you must use absolute paths with `docker run`.
---
title: Installation and Running
permalink: /extend/component/docker-tutorial/setup/
redirect_from:
- /extend/docker/tutorial/setup/
---

* TOC
{:toc}

To work with Docker, you need to have it installed. If you do not have a computer with a Docker machine, you can obtain a
cheap [hosted server](https://marketplace.digitalocean.com/apps/docker). You can also
run everything locally. To install a Docker machine on Win/Mac,
use [Docker Community Edition](https://hub.docker.com/search?type=edition&offering=community). For
other systems, see the [documentation](https://docs.docker.com/install/).

## Getting Started
To test that everything is running correctly, start with an example
from the Docker [documentation](https://docs.docker.com/get-started/).
Run the following commands on the command line:

docker run hello-world


or

docker run docker.io/library/hello-world:latest

If this works, use any image published in any Docker
registry, e.g. [Docker Hub](https://hub.docker.com/), [Quay](https://quay.io/) or [AWS ECR](https://aws.amazon.com/ecr/).
Note that in some configurations, you may need to use `sudo` to run `docker`. If you run into problems, consult the
official troubleshooting ([Windows](https://docs.docker.com/docker-for-windows/troubleshoot/), [Mac](https://docs.docker.com/docker-for-mac/troubleshoot/)).

## Generally Useful Commands
- [`docker run`](https://docs.docker.com/engine/reference/run/): Run an
image (create a container and run the command in `ENTRYPOINT` or `CMD` section of **Dockerfile**).
- [`docker build`](https://docs.docker.com/engine/reference/commandline/build/): Build
an image (execute instructions in **Dockerfile** and create a runnable image).
- [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/): Pull
a newer version of an image (force update of the cached copy).

## Sharing Files
Sharing files between the host OS and the container is done using the `--volume` parameter of the `docker run` command:

docker run --volume=/hostPath/:/containerPath/ imageName

Do not use any spaces around `:`. The details of file sharing are somewhat
[dependent on the host OS](https://docs.docker.com/storage/volumes/) you are using.
There is a very
useful [guide for Rocker image](https://github.com/rocker-org/rocker/wiki/Sharing-files-with-host-machine), which
describes all the sharing options in great detail.

On Linux systems, where Docker runs natively, there is really not much to think about. The only things that can bite
you are file and directory permissions. Keep in mind that files created within the running Docker container on
a mounted host volume will be created with the permissions of users of that container (not the host OS).

On Mac OS, follow the [official example](https://docs.docker.com/docker-for-mac/osxfs/).
The limitation is that your host path must be in a shared directory (e.g., `/Users/`).

On Windows, follow the [official example](https://docs.docker.com/docker-for-windows/#shared-drives).
The limitation is that you must use absolute paths with `docker run`.
4 changes: 1 addition & 3 deletions extend/component/tutorial/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,9 +236,7 @@ We offer free hosting of your Docker images in the **[Amazon Container Registry
All repositories in AWS ECR are private. When you create your component using the method shown above, we
have just provisioned you with the Docker image hosting and you do not need to worry about it any more.

We also support the DockerHub and Quay.io registries, both public and private. However, as they are more prone to
outages and beyond our control, we recommend using our reliable AWS ECR. Use DockerHub or Quay.io only if you,
for instance, want the image to be public.
AWS ECR is the **only supported registry** for Keboola components, providing reliable hosting for all Docker images.

## Summary
You have just created your own Keboola component. Although it does not do much, it shows the easiest path
Expand Down
4 changes: 1 addition & 3 deletions extend/generic-extractor/publish.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,7 @@ have created the component, edit it, and fill in the following details:
- **Region** -- leave empty
- **UI options** --- set to `genericTemplatesUI`

For a list of available tags, see the [Generic Extractor GitHub repository](https://github.com/keboola/generic-extractor/) or
[Generic Extractor Quay repository](https://quay.io/repository/keboola/generic-extractor/), both of which contain the same tags
as the above AWS ECR repository. It is also possible to use the `latest` tag, which points to the highest available tag. However,
For a list of available tags, see the [Generic Extractor GitHub repository](https://github.com/keboola/generic-extractor/). It is also possible to use the `latest` tag, which points to the highest available tag. However,
we recommend that you configure your component with a specific tag and update it manually to avoid problems with breaking changes
in future Generic Extractor releases.

Expand Down
4 changes: 2 additions & 2 deletions extend/generic-extractor/running.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,11 @@ configuration you want to execute. For example:

Then run Generic Extractor in the current directory by executing the following command on *nix systems:

docker run -v ($pwd):/data quay.io/keboola/generic-extractor:latest
docker run -v ($pwd):/data 147946154733.dkr.ecr.us-east-1.amazonaws.com/developer-portal-v2/ex-generic-v2:latest

or on Windows:

docker run -v %cd%:/data quay.io/keboola/generic-extractor:latest
docker run -v %cd%:/data 147946154733.dkr.ecr.us-east-1.amazonaws.com/developer-portal-v2/ex-generic-v2:latest

You should see:

Expand Down
2 changes: 1 addition & 1 deletion overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The following chart shows how Keboola is structured. All Keboola parts are brief
Everything you can do in the Keboola UI can be done programatically using the API of the corresponding component.
All of our components have API documentation on [Apiary](https://keboola.docs.apiary.io/#) and
most of them have a public [Github repository](https://github.com/keboola/).
Our Docker components are built either on [DockerHub](https://github.com/keboola/), [Quay](https://quay.io/organization/keboola) or privately on [AWS ECR](https://aws.amazon.com/ecr/).
Our Docker components are built on [AWS ECR](https://aws.amazon.com/ecr/).

This means that there are virtually **endless possibilities of what can be done with Keboola programmatically**.

Expand Down