Useful tools for development & data science

I have compiled this list over the years to give some credit to projects that have made my life easier.

Basics

• Git, especially the following commands:
  • git add -p
  • git cherry-pick
  • git rebase -i HEAD~N
  • git blame
• ZSH, and more specifically Oh my Zsh and its Git plugin
• When it comes to choosing an IDE, I usually use VSCode (including its browser-based alternative), but I have used SublimeText & PyCharm in the past
• My development workflow usually involves experimenting in Jupyter, quickly moving code to python packages in VSCode and importing them using pip install -e and the autoreload extension.

Python code

• For linting: Pylint & Black
• For testing: Pytest and unittest, especially for its mock submodule
• For logging: python’s built-in logging module (here is a snippet for a quick setup)

Data science

• Let’s start with the obvious: scikit-learn, pandas, NumPy, PyTorch & TensorFlow
• Kedro & Metaflow provide interesting frameworks when building applied ML systems
• MLflow for simplified experiments logging

Others

• I like to bundle my projects using make and / or Docker to make them easier to use by everybody