Dis.co works hard to stay on the cutting edge of serverless computing solutions. Our team is constantly building new features and updates into the platform to
In the first blog post of this series, we reviewed the history of graphics rendering and how computers never seem to be able to outpace the rapid growth of computing demands. More specifically, the only way people have ‘beaten’ the deadlines previously was to create clusters of computers and building infrastructures (a render farm) to scale the solution at a significant cost.
Today, one might ask how can we set up hundreds of machines to run the rendering jobs reliably and efficiently without a huge budget and a large team behind the scenes? More importantly, how is it possible to automate the entire process without having someone constantly managing these machines? Finally, how can we deploy our applications immediately without buying any additional hardware? In this post, we will introduce the first step in a solution which is using a tool called Docker to create containers that help us to deploy solutions quickly.
Docker is an OS-level virtualization management system that allows developers to create a portable virtual container that runs applications, such as Blender, on top of the operating system. Instead of configuring a machine with a virtual machine, which often has a larger image size and deployment time, the virtual container runs on top of the virtual machine and can support multiple applications simultaneously. Today, Docker is a well-known tool that is already used extensively and you can try it for free.
The Blender-Docker container is like a ‘master’ copy for the application. It contains all the dependencies, binaries of Blender and all the other tools required to run jobs. In comparison to a virtual machine, it is much lighter weight (in terms of memory and storage) and also much faster to deploy (see What is a Container?).
Installing Docker Desktop
Let’s start by installing Docker on your machine. You can download Docker Desktop from this link. Please note that a user account is required and you can create one for free on the Docker Hub website.
To install on a Mac, refer to this link.
To install on Windows, refer to this link.
To install on Linux (Ubuntu, CentOs or others), refer to this link.
Once you have downloaded and installed Docker, you can test your installation by running the docker command on the Terminal in Mac or Linux, or on a command-line tool in Windows.
Building a Blender-Docker Container
Now, we are ready to build the docker container/image that we will use to build and deploy our Blender applications on different machines. Docker provides a simple tool to create a customized container using a build script (See Best practices for writing Dockerfiles). Once you understand the flow, it’s easy to create, update, and release these images with full automation. For our use case, here’s a sample build script for running Blender 2.81a on Docker. You will find a copy of the script here in the repository.
FROM python:3.7-slim-buster ENV DEBIAN_FRONTEND noninteractive #get latest python & blender related dependencies RUN apt-get update && apt-get install -y --no-install-recommends apt-utils python3 python3-virtualenv \ python3-dev python3-pip libx11-6 libxi6 libxxf86vm1 libxfixes3 libxrender1 unzip wget bzip2 \ && rm -rf /var/lib/apt/lists/* #get the dependencies for the script RUN mkdir -p /local/ RUN pip install requests pathlib #get the blender 2.81a and setup the paths RUN cd /tmp && wget -q https://mirror.clarkson.edu/blender/release/Blender2.81/blender-2.81a-linux-glibc217-x86_64.tar.bz2 \ && tar xf /tmp/blender-2.81a-linux-glibc217-x86_64.tar.bz2 -C /usr/bin/ && rm -r /tmp/blender-2.81a-linux-glibc217-x86_64.tar.bz2 #copy the shared lib for blender RUN cp /usr/bin/blender-2.81a-linux-glibc217-x86_64/lib/lib* /usr/local/lib/ && ldconfig # Entry point for dis.co WORKDIR /local/ ENTRYPOINT ["python", "-u"]
In this build script, it first uses the command “FROM python:3.7-slim-buster”. This line defines which default Docker image we will be starting from as the baseline. Particularly, we selected the Docker image with Python 3.6+ pre-installed on Debian 10. This provides us a smaller footprint and thus speeds up the deployment process. Of course, we can switch to other images based on the needs or application preferences. Additionally, you can preview all the containers that are already available in DockerHub and it’s simple to customize such. Please note you must log in to see that page.
Then, the build script executes a set of commands with the “RUN” command that installs the latest python packages and other dependencies for Blender. More importantly, the build script downloads the Blender application and also extracts the packages.
Lastly, the build script sets up the entry point of the container to run Python by default.
To run this build script, we execute the command as follows and a new Docker image will be generated locally.
docker build -t raymondlo84/disco_blender -f base.dockerfile .
Also, you can replace raymondlo84 with your own Docker ID if needed for backup or other experimental purposes.
By running the build command, there will be a local Docker image to test our Blender projects. At any time you can check the repository, tag, image ID, and the size of the Docker image by using the following command.
And Docker will return a list of images that installed locally.
As you can see, the docker image is approximately 924MB which is much more compact than most of the virtual machine images. Now, we can run the Bash command inside the docker image, and test out the system locally in the next step.
Running Blender inside a Docker Image
Now, with the Docker image created, we are now ready to test out the system. First, we will execute the following command to enable the interactive mode in Docker. That’s we will be creating a bash command-line interface with the Docker image, and thus allowing us to run the command directly. We can think of this similar to ‘ssh’-ing into a machine.
docker run -i -t --entrypoint='/bin/bash' -v ~/Downloads:/media/ raymondlo84/disco_blender
Notice that the parameter input “-v ~/Downloads:/media/“ mounts my local Download folder to the media folder inside the Docker environment. You can treat the media folder as a shared folder or a link between your current filesystem and virtual container’s filesystem. Then, the final parameter “raymondlo84/disco_blender” is the Docker image we created in the last step. In your case, you should replace it with your own Docker Hub repository (i.e., “<dockerhub_id>/disco_blender“).
To test our system, we will run the Class Room demo scene provided by Blender. You can download the demo scene with this link. Once downloaded, we extract the classroom.zip to the local “Download“ directory to ensure that the Docker image can see the project file when we execute any commands inside the Docker image. Additionally, we recommend that you download and install Blender with GUI on your system so you can configure the project files and perform quick local testing.
For instance, the default setting of the Class Room Demo scene is shown above, and it renders the scene at 1080p resolution and then outputs the result as a lossless PNG file.
Then, once you have confirmed everything is setup properly, we will now execute this command inside the Docker to render the same scene.
/usr/bin/blender-2.81a-linux-glibc217-x86_64/blender -b /media/classroom/classroom.blend -x 1 -o /media/output.png -f 1..1
The Blender command has a few key parameters. ‘-b’ allows us to run the Blender in the background (i.e., no GUI, and thus we do not need to attach to any displays). ‘-o’ defines the output path, which in this case will be a png file. Finally, ‘-f’ sets the range of frames that will be rendered from the animation sequence. In this example, the specified command will only render the first frame from the sequence.
Now, on our terminal screen, we see progress updates in real-time along with an estimate of the remaining time and memory usage.
On my local setup, the scene takes approximately 25-30 minutes to render on the MacBook Pro’s 2.8 GHz Quad-Core Intel Core i7. The exact amount of time used depends on thermal throttling or any background tasks. If we render the same scene with a 4K resolution, it will take over 2 hours to complete one frame.
Once the rendering is complete, we can review the final results by opening the output.png file in the local Download directory.
At this stage, we have a fully functional Blender application running on a Docker container that can now be deployed on other machines. The next step is to show how to run this script across hundreds or thousands of machines concurrently – without human intervention. For that, we will look at a new tool called Dis.co, a distributed computing platform that allows you to run jobs in parallel with simple Python scripting.
Here is a preview of the Dis.co workflow for parallelizing Blender jobs across machines with the Docker container. Stay tuned for the next tutorial.
To see how Dis.co can accelerate video rendering for your agency or company, please visit try.dis.co/render.