Installing Python Packages In Your Docker Container¶

  • Workshop Video

  • pip (and a Linux parcel manger) vs anaconda

  • Case Study: Spacy

Note

Connecting to get command line access: ssh wustlkey@compute1-client-1.ris.wustl.edu

Queue to utilise: workshop, workshop-interactive

Group to use: compute-workshop (if role of multiple groups)

Workshop Video¶

pip (and a Linux packet manger) vs anaconda¶

  • pip and conda are the two most popular ways to install python packages. There may be instances where you lot can only discover directions on how to install an application with one tool or the either. In that example, the decision has been fabricated for you lot. In situations where y'all can find both, using pip and and a your Linux packet manager (e.yard., apt-become in Debian and Ubuntu) can yield quicker build times and smaller final container sizes. However, because pip can only install Python packages, yous may find yourself also having to apply your bundle director (i.east., apt-go install -y dependency ) to install non Python dependencies. Conda has the advantage of including non-python dependencies. It's disadvantages are the slower build times and significantly larger final containers.

  • pip install documentation: https://pip.pypa.io/en/stable/reference/pip_install/

  • conda install documentation: https://docs.conda.io/projects/conda/en/latest/commands/install.html

Case Study: Spacy¶

  • spaCy is a free, open-source library for advanced Natural language Processing (NLP) in Python. Here's some sample code we will call script.py from Spacy that iterates over every token in the string How-do-you-do World! . script.py will be created in the aforementioned directory as the Dockerfile.

                                            from                    spacy.lang.en                    import                    English                    # Create the nlp object                    nlp                    =                    English                    ()                    # Created by processing a cord of text with the nlp object                    doc                    =                    nlp                    (                    "Howdy globe!"                    )                    # Iterate over tokens in a Doc                    for                    token                    in                    doctor                    :                    print                    (                    token                    .                    text                    )

  • Pip Example: Installing Spacy with pip

                                            # Python has an official epitome. Alpine and slim are the small versions                    FROM                    python                    :                    three.seven                    .                    seven                    -                    slim                    -                    stretch                    #Installation directions plant at https://spacy.io/usage --no-cache-dir allows i to save space in the final prototype                    RUN                    pip                    install                    --                    no                    -                    cache                    -                    dir                    -                    U                    spacy                    # Copies script.py file in my electric current directory to root in the container                    Re-create                    script                    .                    py                    /

  • Build the image with the image name of spacy and a tag of pip then yous'll be able to compare the image sizes on Docker hub

                                            docker                    build                    -                    t                    dockerhub_username                    /                    spacy                    :                    pip                    .                    docker                    push                    dockerhub_username                    /                    spacy                    :                    pip

  • Conda Example: Installing Spacy with conda

                                            # Conda has an official base image. miniconda3 is the smaller python3 based on Debian stretch                    FROM                    conda                    /                    miniconda3                    RUN                    conda                    install                    -                    c                    conda                    -                    forge                    spacy                    COPY                    script                    .                    py                    /

  • Build, tag, and push the image with a name of spacy and a tag of conda;

                                            docker                    build                    -                    t                    dockerhub_username                    /                    spacy                    :                    conda                    .                    docker                    push button                    dockerhub_username                    /                    spacy                    :                    conda
  • Compare Prototype Sizes

    • Check your Docker hub contour and click on your spacy image. You will come across a conda and pip tag under tags. pip is much smaller and you probable noticed information technology was faster to build.

    ../../../_images/pipvsconda.png
  • Try Running Both Versions on Compute1

    • Which one runs faster and uses less bandwidth?

                                                bsub                      -                      G                      group_name                      -                      Is                      -                      q                      general                      -                      interactive                      -                      a                      'docker(docker_hub_username/spacy:pip)'                      python3                      /                      script                      .                      py                      bsub                      -                      G                      group_name                      -                      Is                      -                      q                      general                      -                      interactive                      -                      a                      'docker(docker_hub_username/spacy:conda)'                      python3                      /                      script                      .                      py