Python: Flask Development on Kubernetes with DevSpace

  • Deployments via Helm, kubectl
  • Dockerfile modification in-memory on execution time
  • Development tools, such as file synchronization, log aggregation
  • Custom hooks are actions that are carried out based on events.
  • Custom commands which you can build complex or lengthy commands into a single sub-command.
  • Custom profiles can be used to change anything in the devspace.yaml and Dockerfile using add, patch, remove, and merge operations. The profiles bring the ability to use different configurations for specific deployment types, e.g., staging, production, testing.

Requirements

Apart from access to a Kubernetes cluster -either remote or local- we need three CLI tools to be installed if it is not already.

Developing with DevSpace

DevSpace depends on its configuration file devspace.yaml to perform all actions. Since we do not have one in place, let's ask DevSpace's help. Running devspace init will analyze the current folder, ask some questions, and create a minimal devspace.yaml file. This configuration file will describe all our deployments, dependencies, hooks, special commands, and so on.

$ devspace init     ____              ____                       
| _ \ _____ __/ ___| _ __ __ _ ___ ___
| | | |/ _ \ \ / /\___ \| '_ \ / _` |/ __/ _ \
| |_| | __/\ V / ___) | |_) | (_| | (_| __/
|____/ \___| \_/ |____/| .__/ \__,_|\___\___|
|_|


? How do you want to deploy this project? helm: Use Component Helm Chart [QUICKSTART] (https://devspace.sh/component-chart/docs)

? How should DevSpace build the container image for this project? Create a new Dockerfile for this project

? Select the programming language of this project python


[info] DevSpace does *not* require pushing your images to a registry but let's assume you wanted to do that (optional)

? Which registry would you want to use to push images to? (optional, choose any) Use hub.docker.com => you are logged in as leventogut
[done] √ Great! You are authenticated with hub.docker.com

[info] Configuration saved in devspace.yaml - you can make adjustments as needed
[done] √ Project successfully initialized

You can now run:
- `devspace use namespace` to pick which Kubernetes namespace to work in
- `devspace dev` to start developing your project in Kubernetes
- `devspace deploy -p production` to deploy your project to Kubernetes
- `devspace -h` to get a list of available commands
$ kubectl config use-context docker-desktop 
Switched to context "docker-desktop".
$ devspace use namespace default
#!/usr/bin/env python
from flask import Flask

app = Flask(__name__)



@app.route("/")
def hello_world():
return "<p>Hello, World!</p>"


if __name__ == '__main__':
app.run(debug=True, use_debugger=True, use_reloader=False)
$ devspace dev[warn]   Deploying into the 'default' namespace is usually not a good idea as this namespace cannot be deleted

[info] Using namespace 'default'
[info] Using kube context 'docker-desktop'
[info] Execute 'helm upgrade devspace-flask /Users/logut/.devspace/component-chart/component-chart-0.8.1.tgz --namespace default --values /var/folders/3h/tq577p717mdccgjpcgtcvqv80000gn/T/089143002 --install --kube-context docker-desktop'
[info] Execute 'helm list --namespace default --output json --kube-context docker-desktop'
[done] √ Deployed helm chart (Release revision: 2)
[done] √ Successfully deployed devspace-flask with helm

#########################################################
[info] DevSpace UI available at: http://localhost:8090
#########################################################

[done] √ Port forwarding started on 5000:5000 (default/devspace-flask-5f785f7fdd-6rx5f-devspace)
[0:sync] Waiting for pods...
[0:sync] Starting sync...
[0:sync] Sync started on /Users/logut/dev/loft/devspace-flask <-> . (Pod: default/devspace-flask-5f785f7fdd-6rx5f-devspace)
[0:sync] Waiting for initial sync to complete
[info] Opening 'http://localhost:8080' as soon as application will be started (timeout: 4m0s)
[info] Opening shell to pod:container devspace-flask-5f785f7fdd-6rx5f-devspace:container-0
Installing Python Dependencies
Requirement already satisfied: click==8.0.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (8.0.1)
Requirement already satisfied: Flask==2.0.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (2.0.1)
Requirement already satisfied: itsdangerous==2.0.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (2.0.1)
Requirement already satisfied: Jinja2==3.0.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (3.0.1)
Requirement already satisfied: MarkupSafe==2.0.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (2.0.1)
Requirement already satisfied: Werkzeug==2.0.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (2.0.1)
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.2; however, version 21.2.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

____ ____
| _ \ _____ __/ ___| _ __ __ _ ___ ___
| | | |/ _ \ \ / /\___ \| '_ \ / _` |/ __/ _ \
| |_| | __/\ V / ___) | |_) | (_| | (_| __/
|____/ \___| \_/ |____/| .__/ \__,_|\___\___|
|_|

Welcome to your development container!

This is how you can work with it:
- Run `python main.py` to build the application
- Files will be synchronized between your local machine and this container
- Some ports will be forwarded, so you can access this container on your local machine via localhost:


Image ImageSelector LabelSelector Ports (Local:Remote)
imagerepo-repo1/app 5000:5000


root@devspace-flask-7c4bd546-nfmzf-devspace:/app#
root@devspace-flask-7c4bd546-nfmzf-devspace:/app# python app.py * Serving Flask app 'app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 276-822-672
  • Using Helm, it deployed our application to the Kubernetes cluster by using a component chart.
  • Started port forwarding on port 5000 so that we can access the application as it is running locally.
  • Set up the synchronization of the local folder into the container using the sync feature.
  • Parsed the requirement.txt file and install all dependencies.
  • Using the dev.replacedPods feature, it replaced the original container's image with a development one.
  • Opened a shell into the application container. Note that this is done by changing the CMD instruction in the Dockerfile, so you need to start the application yourself, as shown above.

Adding Cache (Memcached)

Let’s add a caching solution to our Flask application. This will allow us to configure devspace.yaml. Following is the snippet we need to add to the deployments section of the devspace.yaml. This snippet defines a deployment using helm with Bitnami repository and Memcached chart. We also specify the chart version.

- name: memcached
helm:
componentChart: false
chart:
name: memcached
version: 5.12.0
repo: https://charts.bitnami.com/bitnami
vars:
- name: CACHE_MEMCACHED_SERVERS

Adding Environment Values to a Deployment

To add the environment value to our app deployment pod, we need to define them in the deployments section; you only need the env section to be added to the existing devspace.yaml.

- name: devspace-flask
# This deployment uses `helm` but you can also define `kubectl` deployments or kustomizations
helm:
# We are deploying the so-called Component Chart: https://devspace.sh/component-chart/docs
componentChart: true
# Under `values` we can define the values for this Helm chart used during `helm install/upgrade`
# You may also use `valuesFiles` to load values from files, e.g. valuesFiles: ["values.yaml"]
values:
containers:
- image: ${IMAGE} # Use the value of our `${IMAGE}` variable here (see vars above)
env:
- name: CACHE_MEMCACHED_SERVERS
value: $!{CACHE_MEMCACHED_SERVERS}
service:
ports:
- port: 5000
$ devspace list vars
? Please enter a value for CACHE_MEMCACHED_SERVERS memcached:11211

Variable Value
CACHE_MEMCACHED_SERVERS memcached:11211
IMAGE imagerepo-repo1/app

Flask Cache Settings

Following is the change we made to enable caching on the / path. First, we import a caching module, also configuring this module for cache servers; as you noted here, we are using an environment variable, the one that we have given in the sections before; next, configure the application for the cache; lastly, we add cache decorator to our route function.

diff --git a/app.py b/app.py
index d26f024..f840dc3 100755
--- a/app.py
+++ b/app.py
@@ -1,11 +1,16 @@
#!/usr/bin/env python
from flask import Flask
+from flask_caching import Cache
+import os

-app = Flask(__name__)
+cache = Cache(config={'CACHE_TYPE': 'MemcachedCache','CACHE_MEMCACHED_SERVERS': [os.environ['CACHE_MEMCACHED_SERVERS']]})

+app = Flask(__name__)
+cache.init_app(app)


@app.route("/")
+@cache.cached(timeout=50)
def hello_world():
return "<p>Hello, World!</p>"


if __name__ == '__main__':
app.run(debug=True, use_debugger=True, use_reloader=False)
#!/usr/bin/env python
from flask import Flask
from flask_caching import Cache
import os

cache = Cache(config={'CACHE_TYPE': 'MemcachedCache','CACHE_MEMCACHED_SERVERS': [os.environ['CACHE_MEMCACHED_SERVERS']]})

app = Flask(__name__)
cache.init_app(app)


@app.route("/")
@cache.cached(timeout=50)
def hello_world():
return "<p>Hello, World!</p>"


if __name__ == '__main__':
app.run(debug=True, use_debugger=True, use_reloader=False)
root@devspace-flask-7c4bd546-nfmzf-devspace:/app# python app.py* Serving Flask app 'app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 276-822-672
root@devspace-flask-7c4bd546-nfmzf-devspace:/app# curl http://127.0.01:5000<p>Hello, World!</p>
127.0.0.1 - - [21/Sep/2021 11:10:13] "GET / HTTP/1.1" 200 - 127.0.0.1 - - [21/Sep/2021 11:10:15] "GET / HTTP/1.1" 200 -

Checking Cache Stats

To ensure our cache is working as expected, let’s telnet to it from the app container and list stats. You can install telnet with the apt install -y telnet command.

root@devspace-flask-5f785f7fdd-6rx5f-devspace:/app# telnet memcached 11211Trying 10.108.20.56...
Connected to memcached.default.svc.cluster.local.
Escape character is '^]'.
stats items
STAT items:2:number 1
STAT items:2:number_hot 0
STAT items:2:number_warm 0
STAT items:2:number_cold 1
STAT items:2:age_hot 0
STAT items:2:age_warm 0
STAT items:2:age 26
STAT items:2:mem_requested 101
STAT items:2:evicted 0
STAT items:2:evicted_nonzero 0
STAT items:2:evicted_time 0
STAT items:2:outofmemory 0
STAT items:2:tailrepairs 0
STAT items:2:reclaimed 1
STAT items:2:expired_unfetched 0
STAT items:2:evicted_unfetched 0
STAT items:2:evicted_active 0
STAT items:2:crawler_reclaimed 0
STAT items:2:crawler_items_checked 0
STAT items:2:lrutail_reflocked 0
STAT items:2:moves_to_cold 3
STAT items:2:moves_to_warm 2
STAT items:2:moves_within_lru 1
STAT items:2:direct_reclaims 0
STAT items:2:hits_to_hot 2
STAT items:2:hits_to_warm 1
STAT items:2:hits_to_cold 4
STAT items:2:hits_to_temp 0
END

Create a New Endpoint

First, send a couple of requests to the /ping endpoint, validate the response is 404; then edit the app.py file and add the following function with route decorator:

@app.route("/ping")
def ping():
return "pong"
root@devspace-flask-7c4bd546-nfmzf-devspace:/app# python app.py
* Serving Flask app 'app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 276-822-672
127.0.0.1 - - [21/Sep/2021 11:29:44] "GET /ping HTTP/1.1" 404 -
127.0.0.1 - - [21/Sep/2021 11:29:49] "GET /ping HTTP/1.1" 404 -
* Detected change in '/app/app.py', reloading
* Restarting with stat
* Debugger is active!
* Debugger PIN: 276-822-672
127.0.0.1 - - [21/Sep/2021 11:29:50] "GET /ping HTTP/1.1" 200 -
127.0.0.1 - - [21/Sep/2021 11:29:51] "GET /ping HTTP/1.1" 200 -

Deploying to Production

DevSpace is not only a development tool. It can be used to deploy to staging, production, and even with CI/CD pipelines. With the Profiles feature, you can create different deployment settings for your application and its dependencies. Changes include Dockerfile changes, devspace.yaml changes, and so on. You can find a production profile in your initial devspace.yaml You can select a profile or multiple profiles you want to use as follows:

$ devspace deploy -p production

Troubleshooting with Devspace

Logs

By default, DevSpace drops you into the shell of the application container so you can run any command and troubleshoot errors. You can also change this behavior so that logs from all or some (filtering) containers will be streamed.

$ devspace logs
? Select a container memcached-6c775fbd9d-hwbhf:memcached
[info] Printing logs of pod:container memcached-6c775fbd9d-hwbhf:memcached
memcached 14:11:52.97
memcached 14:11:52.97 Welcome to the Bitnami memcached container
memcached 14:11:52.97 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-memcached
memcached 14:11:52.98 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-memcached/issues
memcached 14:11:52.98
memcached 14:11:52.98 INFO ==> ** Starting Memcached setup **
memcached 14:11:52.99 INFO ==> Initializing Memcached

memcached 14:11:53.00 INFO ==> ** Memcached setup finished! **
memcached 14:11:53.00 INFO ==> ** Starting Memcached **

Entering into Containers

You can use DevSpace to control the deployed environment. One aspect is quickly entering the containers. Not specifying the container name allows you to select from a list.

$ devspace enter? Which pod do you want to open the terminal for?  [Use arrows to move, type to filter]
> devspace-flask-7c4bd546-nfmzf-devspace:container-0
memcached-6c775fbd9d-hwbhf:memcached
$ devspace enter -c container-0[info]   Opening shell to pod:container devspace-flask-7c4bd546-nfmzf-devspace:container-0
root@devspace-flask-7c4bd546-nfmzf-devspace:/app#

Running Commands within a Container

Using the same subcommand enter, we can run commands within the container directly.

$ devspace enter -c container-0 -- cat requirements.txt[info]   Opening shell to pod:container devspace-flask-7c4bd546-nfmzf-devspace:container-0
click==8.0.1
Flask==2.0.1
Flask-Caching==1.10.1
itsdangerous==2.0.1
Jinja2==3.0.1
MarkupSafe==2.0.1
python-memcached==1.59
six==1.16.0
Werkzeug==2.0.1
$ devspace enter -c memcached -- ps aux[info] Opening shell to pod:container memcached-6c775fbd9d-hwbhf:memcached
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
1001 1 0.0 0.0 409280 3684 ? Ssl Sep20 0:44 memcached -u me
1001 58 0.0 0.0 7644 2700 pts/0 Rs+ 12:40 0:00 ps aux

Clean Up

DevSpace provides a subcommand that deletes all the deployments and the resources (except PV/PVC) easily so you can remove the development environment with ease.

$ devspace purge

Conclusion

We have seen and implemented some features of DevSpace that will help developers. Seamless deployment, ease of development are powerful features. Also, being able to distribute this configuration to the team is very easy to do.

Further Reading

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Loft Labs

Loft Labs

637 Followers

>> www.loft.sh << Build Your Internal Kubernetes Platform With Virtual Clusters, Namespace Self-Service & Secure Multi-Tenancy