Once you’re done, you’re ready to go! Materials To run Airflow on Kubernetes you need 5 tools: Docker, Docker Compose, KinD, Helm and Kubectl. Without Kubectl, you won’t be able to get the logs of your PODs, debug your errors or check your nodes. As you know, Helm allows you deploy applications on Kubernetes whereas Kubectl allows you to run commands against your Kubernetes Cluster.
KUBERNETES AIRFLOW DOCKER INSTALL
In addition to Docker and KinD, you obviously need to install Helm as well as Kubectl. If you know MiniKube, Kind is pretty similar now (It wasn’t the case a few years ago). It was primarily designed for testing Kubernetes but it is perfect to have a quick local development environment to experiment applications that run on top of Kubernetes. KinD is the second tool you need as it allows you to set up and run a Kubernetes cluster using Docker container “nodes”. Now you may ask: “How can I create a Kubernetes cluster with Docker?” Well, say hello to KinD! You are going to have multiple Docker containers where each container will represent a Kubernetes node. Why? Because the Kubernetes cluster will be created in Docker. Therefore, there are some prerequisites before jumping into the practice part.įirst, you need to install Docker and Docker Compose. In this very hands-on focused tutorial you are going to create a local multi-node Kubernetes cluster to deploy and run Airflow on it. Deploy your DAGs on Kubernetes with GitSync.Install dependencies with Airflow on Kubernetes.
0 kommentar(er)
