Virtual environment management and IDE
2.1 Why Python virtual environments
Although one can run Python command simply as other programs, the base Python comes with limited functionality. For exmaple, only using base Python, we cannot access functions defined in the Numpy module.
Python versions and module versions require compatability. Incompatibility can occur when, for example,
- in Python 3.8+, you use async or await as variable names.
- older matplotlib modules meets support newer Numpy. (Try matplotlib==2.2.2 and numpy==1.20)
- DeprecationWarnings
We sometimes need to work with older versions of our code (recall the code you wrote 4 years ago). Other times, we might just want to use the most updated version of Python modules. To resolve the incompatibility issues, we create self-contained Python environments.
pic source: https://www.dataquest.io/blog/a-complete-guide-to-python-virtual-environments/
2.2 Using venv to manage environments
The venv documentation is here.
| purpose | command (in terminal) |
|---|---|
| create venv | ./python.exe -m venv ./newenv |
| go into newenv | .\newenv\Scripts\Activate.bat |
| upgrade pip | .\python.exe -m pip install --upgrade pip |
2.3 Using conda to manage environments
Difference between venv+pip and conda.
It is a good idea to get familiarized with the coding environment: the interplay of the Python interpreter, the supporting packages managed by venv, and the code that calls them.
Using pip, we have the environemnt managed almost all by ourselves, including the locations. Using Anaconda, a big heavy Python distribution that automatically have 250+ packages installed.
💡 What's your choice?
In general, when you set up your computer, make a directory (or have conda do it for you) to store your Python "workers". Store you code and data independently of that environment. Call Python to do its job when you need it.
| purpose | command (in Anaconda Prompt) |
|---|---|
| create a named conda env | conda create --name newenv |
| create a conda env by loc | conda create --prefix ./envs |
| go into newenv1 | conda activate newenv |
| install a package | conda install numpy |
Takehome message
1. venv is part of Python standard library.
2. Check pip when installing Python.
3. conda is a chamberlain taking care of your Python environments.
2.4 Writing code
Choose whatever you like. Pycharm, spyder, VS Code, etc. Note the combination #%% is handy to execute parts of code.
Separate from the Python virtual environment, you need organize your code properly. For example:
1. For Linux servers, use source activate ↩