How to select an IDE for Data Science?

Over the seven jewelled hills beyond the seventh fall, in the cottage of the seven dwarfs dwells the first programmer, and does her coding on a paper…

Taking it seriously, before IDEs were introduced, programmers used text editors such as Notepad to write code. To save their progresss, they stored that code using programming language extensions such as .php for PHP files and similar. However, these text editors were very simple and as you might imagine, they lacked some features. Can you imagine how difficult and time-consuming writing, debugging, and running code needed to be? Would you still choose this path in similar reality?

Fortunately we don’t need to worry about this anymore. These days we can benefit from development of the IDE, which includes some core functionality that extends beyond text editing, such as:

Syntax Highlighting
Auto Completion
Easier Debugging

The truth is that you can code in almost any software, from prompt command to Windows notepad, but you may also want a proper programming environment which combines coding facility with a debugging environment. What’s interesting to know, although that amount of options being developed, there are still professionals choosing notepads anyway!

You can check this fun fact in the results of the Stack Overflow Developer Survey made in 2020 so not that long ago.

But first…

What exacly do you mean with IDE?

IDE stands for Integrated Development Environment. It’s a coding tool which allows you to write, test, and debug your code in an easier way, as they typically offer code completion or code insight by highlighting, resource management, debugging tools… so all in all, just making things simpler for you! And even though the IDE is a strictly defined concept, it’s starting to be redefined as other tools such as notebooks start gaining more and more features that traditionally belong to IDEs. For example, debugging your code is also possible in Jupyter Notebook. Towards this topic will move on in the later part of an article.

So why would choose a traditional IDE instead of a notepad?

The answer would be practicality. For instance, imagine that you are coding in any text editor like well known Windows notepad. When you finish, you’ll need to run your code. You can’t execute your program in a text editor like this, so you must use a prompt command to do it. Rather than use two different programs, wouldn’t better have it all in just one place? That’s what an IDE is ready for.

Referring again to the Stack Overflow Developer Survey you can investigate what options do we currently have in the market. Some of them you for sure at least heard about, some of them you’ve might already use and have your own reflections.

But with so many IDEs and no one-size-fits-all solution, how do you know which one to choose? Let me cover some of the best practices for selecting an IDE so that you can make an informed judgment and decision when deciding which one is right for your project.

Why is the right choice of IDE so important?

Since the editors differ, with such elements as:

Code formatting (to a more human-friendly form!)
Language syntax recognition
Prompting language elements to be used
Enhancements regarding working with files
Easier possibility of collaboration with your team

But to make this choice right you need to consider many elements. Let me guide you with several most important factors before you’ll decide to test any tool in particular.

Your IDE should be compatible with your operating system

If you’re a Linux user, you’ll want to look for an IDE compatible with Linux. If you’re a Windows or macOS user, you’ll want to ensure your IDE is also compatible with those operating systems. It’s important to note that IDEs are not limited to operating systems: Pythonista works on all platforms and does not require installing any additional software.

Your IDE needs to be compatible with your programming language(s)

In addition to Python, many IDEs are compatible with other programming languages like C++, Java or R. If you already know how to code in one language and want to learn another, then an IDE can be beneficial for this purpose. For example, suppose you already know Python very well and now aim to learn R. In that case, an IDE can help you understand its syntax while reminding you of your previous language(s) commands. Surprised? Yeah, you are not tied to RStudio if you are an R user. What’s more, you can use RStudio to code in Python!

Additionally, some IDEs support multiple languages at once. This is known as a multi-language environment (or “MLE”). When learning new languages in MLEs, such as PyCharm or VSCode, it’s easy for developers who already know several different languages to make mistakes. It is because they think about them differently than usual due to their familiarity with other similar syntaxes within those same MLEs (e.g., writing functions in R vs Python). You can think about it similar way we struggle with words knowing several foreign languages – was “Buenos días” Italian or Spanish?

Your IDE should be compatible with Data Science libraries and frameworks

In addition to being compatible, the IDE should also support your development environment. You can have different needs for creating a webpage than developing machine learning model.

For example, the ideal data science IDE will:

Support your programming language and libraries
Provide support for data science libraries, such as Pandas
Provide support for machine learning libraries, such as TensorFlow
Provide support for data visualization libraries
Provide support for version control, such as Git

Your IDE should be compatible with Docker

We shouldn’t forgot about Docker images which are so beneficial! If your IDE does not support Docker, it is not a good fit for data science. Whether it’s the first time you hear this term, Docker is an open-source platform for managing containers. It allows you to create a virtualized environment for your code and copy it across machines, which can be helpful if you are on different operating systems or using different versions of Python or R. If the IDE supports Docker containers, then you will be able to clone them from one computer to another without having to reinstall all of your tools or libraries every time you change computers.

List of most popular IDEs for Data Science

As you can see from the previous paragraph, you have already several factors to consider, but there even are more: the possible learning curve, price or built-in/downloadable features…. you also might want to take into account the possibility to visualize and report on your results, or how easy a certain the environment is to collaborate with others. All of this make this task even harder!

To support you in making the first step (and don’t waste much time procrastinating instead of moving into actual coding practice!), let me introduce you some of the most popular IDEs for data science.

RStudio

If R is your native programming language, probably I don’t need to introduce you to this IDE! RStudio is amazing IDE for development in R. It’s a cross-platform means far fewer nitpicky details to remember when going from Windows to Mac or vice versa. The software works the same on all platforms.

I have heard that the creators also made it possible to code within Python languages in this tool but to be honest, I haven’t tested this opportunity yet. For interested users, I can recommend you a reliable guide from our beloved Data Professor available here.

As every IDE, it has its pros and cons. Some of the advantages I see:

Viewing larger data and plots without writing to files, this becomes easy when you have need to print multiple plots and do enhancements before saving.
It gives us the picture of what is happening. For example, it might seem simple but if I can see variables that are created in the environment, I feel that I can optimize the program better.
Easy code debugging
Nice structure thanks to saving your work into projects
Friendly environment for package development
RMarkdown and knitr in RStudio is a real step forward in documenting analyses

If I was to indicate some potential improvements for the tool, I would mention:

It would be nice if R studio had the ability to independently move each of its 4 panes.
I haven’t figured out the exact reason but sometimes RStudio create problem in viewing data. It does not show all the fields (according to Google, probably because its HTML based interface so not all feature of R works here)

When to use RStudio?

I would recommend it definitely to all users who’s R is prior language. To code in Python, I am still using another IDE as I am simply used to another environment but it’s just a matter of your habits and what’s comfortable to you. I couldn’t imagine being experienced in R programming and not knowing RStudio interface as it’s so popular and convenient, especially is R is your first language to learn.

PyCharm

The most popular Python IDE developed by JetBrains. The tool is distributed both in the Community version (free Open Source version) and in the Professional version (paid). PyCharm is probably the most advanced Python IDE. It has many code refactoring tools, graphical debugger, test run, syntax hints, type checking. The IDE also supports git, the most popular version control tool.

With PyCharm, a developer can also run code. Using a linter, it also highlights bugs and inconsistencies with PEP 8 and allows you to automatically format the code. The paid version of PyCharma also has support for creating websites in Django and Flask, a profiler and database tools. PyCharm also allows for relatively simple management of virtual environments.

There are also many plugins and extensions that can make your code work much easier. PyCharm can definitely be called such a great IDE!

When to use PyCharm?

PyCharm is definitely best suited for large projects, such as projects in Django. It’s very often used during Python dedicated learning courses – I have also gained my Python skills on this IDE! Similarly to coding in RStudio if I am only coding in R, I would select PyCharm if I was only coding in Python.

VS Code

Visual Studio Code is a lightweight IDE developed by Microsoft. The greatest strength of the tool, in my opinion, are extensions that allow you to adapt the IDE to virtually any language. If you’re interested in this IDE, I would really recommend you a tutorial guide done by Corey Schafer where he shows how to configure your Python environment in VS Code from scratch! I haven’t found better resource on this topic than his material, it’s worth to keep it in your bookmarks. 🙂

Together with VS Code extensions, it is a tool that works well for refactoring, has syntax hints, a debugging tool and built-in git integration. Of course, we can adapt the IDE to ourselves by adding extensions to facilitate, for example, working with Docker or markdown.

When to use VSCode?

It works especially well for medium-sized programming projects, and projects that use different technologies and languages. It is a much “lighter” IDE than PyCharm, but very often it will be quite sufficient, especially if we do not have a paid version of PyCharm. Personally, it’s the best choice for me if you’re working with several programming languages in your projects.

Jupyter

Jupyter is a development environment designed specifically for working with data. The Jupyter project includes tools such as Jupyter Notebook and JupyterLab.

Jupyter Notebook is a tool for creating and executing code gathered into blocks. The result of block execution is saved right below it, which makes it very easy to work with, for example, many diagrams.

JupyterLab, on the other hand, is an environment that allows you to manage notebooks, code files or even data files (e.g. in CSV format).

After launching the environment, it creates a local server to which we connect via the browser and operate the environment from its level.

When is it worth using Jupyter?

It is incredibly easy to use for beginners. This is because it allows you to both write your code and then test it in the same interface. The interface is extremely intuitive. It features the terminal, text editor, file directory, and console all in a single layout. The interface can also be customized using features such as magic commands or notebook extensions. But this doesn’t mean Jupyter’s usability is limited to the newbies.

Jupyter is, in my opinion, by far the best environment for working with data, especially if we use the pandas library or draw a lot of graphs. If you need to simply experiment before initiating large project, create the proof of concept or develop something smaller, there is no better environment to make it quickly. What’s more notebooks are also perfect as reports.

Other environments using notebooks

I think it’s also worth to mention that notebooks are extremely popular also with GoogleCollab, Azure Databricks (this option allows you to code with more than one language in the same notebook!) and notebooks you can develop in well known Kaggle. All those environments allow you to write and execute arbitrary code through the browser, and is especially well suited to machine learning, data analysis and education.

But wait… aren’t Jupyter Notebooks enough?

Well, let me give you our all time favourite answer – it depends… on the task.

Jupyter Notebooks are designed to be a rapid prototyping and visualization tool, including the ability to embed code, text and programatically generated graphics into a web page, and the ability to share notebooks. It’s not the best idea trying to implement a complex application or library in such environment, but on the other hand I wouldn’t open VSCode to try a Python fragment or small example. It’s great idea to use both but they aren’t interchangeable.

Do you have a quick data processing and visualisation job?

Open Jupyter.

Do you have a complex data processing job or machine learning model based application where the code needs to be shared with other developers or maybe the project/application is complex, probably to be timeframed for months/years?

Use VSCode or another IDE.

Very basically, an IDE is a means to an end, whereas a Notebook is in itself the end product.

Summary

Choosing the right IDE for data science is an important decision. An IDE is a tool that allows you to have access to all of your most frequently used tools and packages without having to switch constantly between them. These tools typically include an integrated code editor, terminal window, file system browser, debugger, etc.

The best way to decide which IDE will work best for your needs is by carefully considering what workflow works best for you and what types of tasks you want the toolset to help guide through completion.

Summing everything up, let me leave yourself with a short table with my subjective selections I would recommend you for the specific purpose. Please bear in mind, that’s it’s just my opinion and each of you needs to find their own most convenient setup and ways of working. If my choice for experimentation is Jupyter notebook it doesn’t mean that you cannot choose VS Code for this purpose or it’s not possible. 🙂

IDE	Quick Experimentation/Proof Of Concept	Big project	Python	R	Multilingual	Working alone	Collaboration with others
RStudio		X		X			X
PyCharm		X	X				X
VS Code		X	X		X		X
Jupyter	X		X	X		X

Comparison of described IDEs

If you want to get familiar with more articles about data scientist’s toolbox which are already on my blog, you can read about experimenting with MLFlow and Azure Databricks Dashboards. 🙂

How to choose your best IDE for Data Science?