How Deep Learning can be used for social good? Interview with Karol Majek, the expert in the field of mobile robotics, self-driving cars and computer vision.

Welcome in 2022!

Can you imagine? I have a feeling that not that long ago I have summarized 2021 and now I am doing same for 2022. Time flies so fast, but it doesn’t mean that it has passed unnoticed. Actually, it gave me so much pleasure to compare my progress between those two years. I need to admit that I am the kind of person, who often underestimates what they have achieved. Regardless of the field in which I would like to develop myself, the process is always more important to me. I tend to accomplish my plans and instead of drawing conclusions, my head is already planning next goals. I know that’s really a dark side of my personality and I tend to work on that. That’s also why I would like to start today post with some summation.

2021 summary

Bright sides

2021 was really good for me. I have completed much more than I initally planned which makes my really happy and proud. I have listed some of my 2021 bright sides:

  • Participating in plenty wonderful courses – I have completed Matt’s Dancho R Shiny Dashboards training, DataWorkshop Natural Language Processing course (first attempts on deep learning) and also certified myself with Azure AI Fundamentals. I have collected so many ideas of how to apply data science broadly and definitely got inspired for future activity!
  • Taking part in some interesting projects in my workplace, being able to learn from inspiring people around me, who have been persistent and patient enought to answer all my questions. 🙂
  • Challenging myself in the role of Scrum Master for my team which I have found so rewarding as I love working not only with data but also with people. Although it will probably never be my primary role, I really appreciate this possibility as it gives my broader perspective on projects and teams.
  • Working a lot on Citizen Data Science community development. One of the things I am most proud about this year is moving from live R programming trainings into self-created e-learning platform. I have created all recordings and material on my own which also was a great experience for me. It’s so rewarding to create something from scratch.
  • Moving my blog to own domain, learned about hosting and increased my experience with wordpress sites, html and css. Although I am 100% conscious that there is still a large place for improvement, I am really satisfied with the new appearance of my site and amount of control I have after the changes.
  • DataScientistDiaryPL instagram creation (channel for Polish community) which has resulted in meeting many wonderful people from the field, also interviews on the blog (like the one I have for you today!).
  • Participating in several inspiring conferences. Some of them I’ve took part remotly (like EARL2021), but I have also managed to participate in InfoShare personally, supporting my leader in his lecture and promoting my workplace.
  • Getting rid of the Junior prefix from my job position. 😀
Junior vs Senior - Album on Imgur

Dark sides

Being online gives so many opportunities. You can meet more people without actually leaving your home, you can learn from variety of resources, participate in remote courses and events. Unfortunately, it can also have a detrimental effect on us. In times when we want to show only our best side of ourselves, very few people talk about it. Scrolling via your Instagram or LinkedIn feed, you can have the reflection that everyone is successful and productive 365 days a year. Everybody seems to be living their dream life. It’s very dangerous as looking just on those successful stories, you can feel that you should do more. And ‘this more’ is never enough.

I must admit that although I am proud of my achievements in 2021, I know that I pushed myself too fast for a moment. How do I know about it? Because when it was time to rest, I was starting to feel guilty. Why should I rest now if I can do more instead? Take another course, write another blog post or read a book. Just as it’s easy to procrastinate, it’s also easy to overact. To learn new thinks, to take new challenges, you need to treat yourself well in meantime. Give some time for reflections over your actions. Sometimes I have felt like this man running with an empty wheelbarrow, having no time to load it, because I wanted to have a feeling that I am doing best from myself.

“Dudes, you are running like crazy and pushing empty wheelbarrows around. Why do you do that?”

“Man, we are so busy there is no time to load the damn things!”

Plans for 2022

That’s exactly why I’ve decided that I don’t want to make any specific New Year’s resolutions this year. It doesn’t mean that I do not have any goals. Of course I do. My head is full of ideas I’d like to cover in 2022. However, I would like to plan wiser my actions, instead of doing more. More doesn’t been better. I am conscious that like every other person on this planet I have just 24 hours every day. I want to use them wisely.

It’s important for me, not to lose this spark, the joy of learning new things and experimenting. I don’t want to feel guilty when I don’t feel productive for a day or too. That’s because I am not a robot (nor AI :D)! I want to be guided primarily by long-term goals, not to accelerate their implementation by force, and to enjoy progress. And that’s what I would like to wish also to all my readers at this point. Always appreciate your efforts. Even the smallest ones you do. Do not compare yourself with others, because you do not know their way. You should compare yourself just with one person. Yourself – the day/month/year before. 🙂

Mobile robotics, self-driving cars and deep learning

How to reconcile all your passions based on Karol Majek example

I am very glad that the interview with Karol fell exactly at the moment of summing up 2021. Comparing his broad experience in the industry with my reflections, gives me so much inspiration. I see that I am just on the beginning of my path and I can use all the knowledge I gain each year to do really amazing stuff. I hope that you’ll agree with myself that my interlocutor is definitely a person who have used all his opportunities and skills properly. Karol is not only a passionate but also a practicioner. He is an expert in many hot fields just in their peak development.

I have been following Karol’s activity on social media since I only appeared on Instagram. It always made me so impressed. Looking at his experience you can really wonder if he ain’t one of those mobile robots like the ones he’s working with! Without further prolonging, I would like to invite you to this interview. You can really inspire yourself, I promise. Not only if self-driving cars, robotics, deep learning or computer vision is interesting for you but also if you’re simply looking for your path.

Sandra: Hello Karol! I am so happy you’ve agreed to this interview. It’s the first time of hosting an expert of computer vision and robotics there. This field is developing so rapidly that I am sure our readers will benefit so much from your experience. You’re definitely a person well known in the Polish community, active in social media, but for the non-Polish visitors of my blog, could you please share a few words about yourself? How have you transferred from automation, vast amount of robotics initiatives into deep learning solutions?

Karol: Hello! Thanks for having me. I have a PhD in the field of engineering and technology in automation, electronic and electrical engineering. I have started my journey in mobile robotics i mechatronics. This is my background – mobile platforms, perception, 3D mapping and of course computer vision. During studies and my PhD I was involved in many R&D projects. First while I was working at Institute of Mathematical Machines where first computers in Poland were developed, and later at NASK – Research and Academic Computer Network which is a big Polish research and development organization.

In mobile robotics, I focused on USARUrban search and rescue. We tried to make mobile robots autonomous, to perform rescue missions in harsh environments, such as simulation of nuclear power plant disaster based on Fukushima. Many challenge scenarios were based on this disaster. Robots are used to build a 3D map of the environment, and find all the missing workers and other objects of potential interest.

Of course, there are also driving cars, which we can consider as large Mobile robots. Since the first DARPA Grand challenge, many roboticists are thinking about creating their own autonomous cars. Of course, I was inspired by this event and tried to navigate towards building such mobile robots. Thus I’ve decided to participate in Udacity self-driving car Nanodegree as one of the first. This program aimed to teach computer vision sensor fusion and small amount of deep learning to finally test your programs on a real self-driving car in California.

Stanley - autonomiczny samochód
Stanley (one of DARPA’s winning cars). Source: Stanford cs221

During the program, I have also participated in the image-based localization challenge organized by Udacity. Our team was the second in this challenge. Soon after, Udacity was forming a team to participate in a self-racing cars challenge. With this team, we were testing our deep learning image-based model to steer a full size vehicle on a racing track.

Of course, we have started with a simulator and were forced to bridge the sim2real gap. We had too many problems to finish a single lap. Fortunately 4 of us managed to participate next year without support of Udacity. Then have we changed our approach from end to end network with image input and acceleration and steering output to a more complex approach. We have decided to use semantic segmentation to detect the road and estimate the track center line. Thanks to this we had an insight into the perception pipeline, our solution was no more a black box, and we were able to drive a few minutes without intervention. Seems like it’s not too much, but we had only like 4 hours with the car.

Sandra: Woah! You have contributed in so many projects since you’ve started your career! I am really amazed. If you need to choose the one which is closest to your heart, which of the mentioned ones would it be? Which has given you the highest amount of satisfaction? Could you please share with us your experiences?

Karol: In our team we want to put our solutions on mobile robotics platforms to the customers, to production. We have started from the mapping purpose to automate the process of building a 3D map of the environment. Such robots can be used in urban search and rescue missions, 3D mapping of buildings, or in the security of large areas such as airports or museums. We were in POLIN museum in Warsaw, driving autonomously with our small robot holding a large precise geodetic 3D Laser scanner. Robot, was able to build a 3D map of this environment automatically. There’s awesome architecture in this place, I pretty much recommend visiting it.

In Łódź airport we had other goal. We prepared a mobile robot was able to compare changes in 3D environment, to find lost baggage automatically. In both cases our ideas were not implemented, but from the research perspective we acquired many awards, for example from BBN – National Security Bureau.

ICARUS project was a European project to integrate ground, aerial and water surface robots with disaster response teams. The project was inspired by the Fukushima disaster. Autonomous robots were meant to be used to build a 3D map from multiple sources to speed up the reaction of disaster response teams. A natural consequence for us was to participate in a robotics hackathon in a real nuclear power plant. We were already two times in Enrich competition which is held in Austria in a Real NPP museum.

I really enjoy project competitions, hackathons, where mobile robot has a real challenge. Autonomous racing head to head, exploring and building 3D map of NPP, search and rescue missions in building without access. I don’t know if there is a single category for such missions but for example an interesting project which I am not involved in is DARPA subterranean challenge focused on exploring underground tunnels autonomously.

Sandra: It’s also really amazing how you appreciate each activity you’ve participated in. So difficult to choose one, as each of them has given you so much satisfaction and value. From my perspective, I would really love to know more about your experiences as a Self-Driving Car Engineer in Udacity. It’s such a hot topic these days! For the readers less familiar with the topic, could you please estimate the current level of this technology worldwide? At what stage are we globally? For someone who doesn’t deal with it on a daily basis, it’s hard to get to the truth when famous people like Elon Musk have the habit of emphasizing the interest by overpromising.

Karol: I was a mentor in the Udacity’s Self-Driving Car Nanodegree program. I was checking progress of multiple students on a weekly basis while helping them solve their problems. Udacity had a pretty good idea at the time to create a course which would help to prepare future self-driving car engineers. It was a perfect timing, since many companies at the time started working on their Self-Driving Car programs such as Waymo (as previously a Google project) or Tesla which 2 years before announced their product named Autopilot (obviously is not an autopilot). Currently, they are surprised that self-driving is not solved and is hard to solve. Elon was claiming that it was a solved problem already, and now he is still saying, as always, that Tesla will release autonomy next year.

We hear those “next year” claims since the few years already. Other companies such as Uber quitted their self-driving programs, since the development requires a lot of resources to fix all the edge cases. Edge cases are situations in which the car fails to drive as expected due to unusual situations.

Tesla’s perception stack is reduced now to vision only, opposite to other companies which are using also radars and lidars. They hope to solve self-driving using only cameras and deep learning. This makes it even harder to fight all the edge cases. From the other hand, Waymo cars are driving too cautious on the streets, and sometimes get stuck even on their geo-fenced area.

How We Hire: The Udacity Self-Driving Car Engineer Nanodegree Program |  Udacity
Udacity’s Self-Driving Car Nanodegree program
Sandra: I know that’s it’s probably question impossible to give exact answer, but if you could estimate, where do you think we can see self-driving cars on Polish streets? How do you see the future of this technology in our country? What risks and challenges do you see?

Karol: In Poland, we have many engineers after the Udacity’s Self-Driving Car Nanodegree course and also after mobile robotics studies at Polish universities. We have people doing such research in Warsaw and Poznań, among other Universities in Poland. Interesting fact is that we had already an autonomous bus from EasyMile driving in Gdańsk in 2019. It is really hard to predict future in self-driving. I don’t want to give bold statements, such as Musk – that anything will happen next year. Even in California, autonomy of cars is pretty limited.

Our streets are similar to other Europe countries, so this shouldn’t be a large problem. I guess that autonomy will be geo-fenced to countries or types of roads and that multi-sensor approach will win – sorry Tesla. A good guess would be minimum 20 years from now because we need to have a first full or geo-fenced autonomy, and then it will be slowly, incrementally deployed in all countries. As you can see, I am trying to be not a super-optimistic-next-year person in my predictions. 

DARPA Grand Challenge, the first autonomous cars challenge was already 15 years ago. Now we are finishing Indy Autonomous – Head-to-head racing F1 cars at 200kph. This event is meant to speed up the development of autonomous vehicles by solving edge case scenarios and catalyzing new technologies.

The other thing is social acceptance of the technology, which is already increased by the Tesla, even if drivers abuse the Autopilot.

Standing still with my self driving car... Future dreams : Meme,  Meta/Reddit, Funny/Humor : r/memes
Sandra: Recently, you have shared on your LinkedIn short summary about yourself 5 years ago and now. According to your words, this amount of time has significantly changed the way you areyou working with deep learning. Could you please explain to the readers how and why? If you had a chance to speak with your younger self, what would you adviseadvice Karol? Do you regret anything?

Karol: So many things changed in deep learning in this time! Hardware accelerators are improved to faster train and deploy models. Deep learning frameworks evolved to provide easier interfaces for researchers, many new neural network architectures emerged, improving the state of the art. Now we prefer results of a model rather than the original human-labeled ground truth in some datasets. We have actually made a full circle in architectures for image recognition, starting from MultiLayer Perceptrons, through Convolutional Neural Networks, Transformers to finally get back to MLPs which are shown to provide decent results.

Deep Learning is now developing rapidly, since it’s not constrained to any domain or event any type of data. You can use similar methods in image, video, 3D, audio, NLP or sequences. This fact and the ability to obtain good results with ease contributed to this rapid development.

If I had a chance to speak with my younger self, I would say that it will be really demanding to be on track with recent advances in the Deep Learning, you don’t need to know every detail. I regret maybe one thing – that I was waiting too long with switching from TensorFlow to PyTorch.

Sandra: Trying to learn every detail is the frequent mistake of beginners, I think. In your post, you have also suggested being happy that the entry level to Deep Learning industry has been lowered. Could you please tell more about it? How your beginnings have been different comparing with 2021 newbies? What would you to people wanting to tie their career path with DL now?

Karol: I remember when I was considering which backend for Keras to use – Theano or TensorFlow. Back then it was already easy to use, compared to Caffe framework. Now the community is large, frameworks are mature and stable. You can easily find tutorials or even find ready-to-deploy models for many tasks.

The other thing is you can find many sources to learn such as official courses on Universities, online-courses by practitioners or video tutorials. I remember when in our team we needed to detect people in from of the robot. During pre-deep-learning era we were using Histograms of Oriented Gradients (HOG) to extract features from an image, which were used to classify if it was a human or not. It was a hand designed approach using a feature extractor and a classifier. Considering limited power of gpus at the time – first steps in deep learning were pretty hard in my case. Now you can use Kaggle Notebooks or Google’s Colab to access GPU resources for simple training sessions.

Career path depends on particular people – some want to start from theory, others need to first start trial and error, then go to theory, some prefer videos, other prefer books. If a person consider postgraduate studies, I would suggest to seriously consider if they really need it. I have heard that sometimes courses can be better than postgraduate studies in this topic. I personally can say – do as many experiments as you want, as you can.

Sandra: Recently, we can read a lot about the AutoML concept. I am curious, what is your opinion on this topic? Should data science go into this direction?

Karol: Automated Machine Learning is a concept to make deep learning available to a broader community. You don’t need to know how the training process works. Your role is to provide data, define the task and specify how fast should it work. We can do everything related to the learning process without your supervision in an automatic process. We can dedicate such tools for companies without ML/DL teams, which want to create a proof of concept or simple solution. If you aim to get the best results using your data, you will probably need to implement a custom neural network instead of AutoML model. So we are going into both directions – researching new architectures and automating the process. AutoML will not eliminate working on new architectures, but will probably make Deep Learning more accessible.

Sandra: Having such great experience with competitions, could you reveal to the readers, how do you select projects you’d like to participate in? What are your criteria? What motivates you the most in your career?

Karol: I am looking for projects related to computer vision and/or mobile robotics constantly, since, I have chosen mobile robotics. I really like the 2D/3D perception, which enables robots to move around in the space. As you know – this is pretty much universal problem in robotics if you can’t change the environment, by integrating for example magnetic tape into the floor to automate navigation.

And I don’t want to participate in projects related to mobile robots equipped with lethal weapons. Currently, I shifted my focus towards deep learning in computer vision in general. In my own company – Cufix – we are working on such topics, and we share our knowledge in online-courses such as our first program – Deep Drive PL Image classification program. Currently, my goal is to share knowledge about deep learning based image analysis, since it gains popularity really fast.

Sandra: Thanks for mentioning the program! That’s something I was going to ask about next. You are also an author of DeepDrive blog about deep neural networks, self-driving cars and robotics from 2018. Could you please share why have you decided to create it and how it evolved during time? What are your tips to stay updated with the most important news in the industry, avoiding FOMO? And what’s more important, how do you manage to reconcile so many initiatives in your life? Be active on social media, write blog posts, run a learning course, develop your knowledge and skills and contribute to the projects at the same time? How to find balance in your own development in the industry which evolves so rapidly?

Karol: In 2018, I finally started the DeepDrive.pl blog after a lot of hesitating. Main goal was to share the knowledge about 3 topics – mobile robotics, neural networks (especially deep learning) and of course autonomous driving. Knowledge sharing was the main idea, why? Because we don’t have many resources in Polish internet. I wanted to fill this gap and also share my experiments. During this few years blog evolved as a place for people to share their experimental results, show deep learning related tutorials etc. I have also created a weekly mailing, where I share my knowledge to the subscribers and my first online training program focused on image classification problem. I hope this will help to scale knowledge sharing. 

You asked also about staying updated, which is quite hard, due to rapidly developments in the field. My solution is to follow a few people and researchers on Twitter who are going through recent publications from CV conferences. I follow also few researchers and organizations on GitHub to easily find their new repositories corresponding to new research papers. Of course, there’s also the fantastic aggregator of the knowledge, PapersWithCode.com which allows you to browse methods and results easily.

Still, you can have FOMO, I don’t have any silver bullet. I try to remember all the time, that there’s no way to know every new method, every new paper, or dataset. The developments now are too fast even in the basic problems such as image classification. In my own company, there are few people who are working with me on creating materials for the course, solutions for clients and blog posts. We are all working constantly to improve our knowledge. We are also trying to shift the balance towards teaching and improving our knowledge and sharing it.

DeepDrive.pl blog about mobile robotics, deep learning and autonomous driving
Sandra: Approaching to the end of our interview, I would like to talk also about your original course – The Image Classification Training Program. You have advertised it as the extensive and comprehensive program in Poland about creating and training deep neural networkds for image recognition. Could you please tell us more about this course? Who have you dedicated this course for? What are the skills that candidates should demonstrate to participate in it? Could you tell us about the benefits participants can gain through the course completion? What kind of excercises will trainees find there?

Karol: Image classification training program consists of 12 modules and guides students from first steps with image processing, through deep learning, building own models, transfer learning, data augmentation, hyperparameter optimization, explainability, self-supervised learning, optimization, quantization and deployment. As you can see, that’s quote a lot of material in one program, which is rare. In the program, there are many notebooks and tasks as graded homework with feedback. The program is for people starting their adventure with computer vision, and people who want to complete their knowledge. For example, someone is working on classification in medical domain and need to learn explainability and deployment methods, or someone who just need to improve their model results or optimize the performance.

Of course, to start you need at least basic knowledge of Python language. It’s good to know matrix algebra it will help a lot while working with 4D tensors. It’s good to start during or after studies in computer science. But if somebody is not sure about, how hard is the program, I give a satisfaction guarantee – 30 day money back, so only satisfied people will stay in the program. After the program, students have already a few demos which they can share on their public GitHub profile to build a portfolio, and a certificate when they finish all graded homework.

Sandra: As far as I know, you have hosted one edition of the course already. What conclusions have it brought to you? What have you learned with this experience? Have everything gone successfully as planned? When do you plan the next edition?

Karol: There’s quite a lot you can actually get from such an online course. Already there are several people happy that they used learned techniques in their work, finishing their university exams, working on their PhD thesis, finding a job in related field. Many good things happened to students in half year since the program started. It makes me so happy. First edition was a pre-sale, no materials existed. We have created everything from stratch during 4 months, based on recent knowledge and our experience in the topic. What’s more, we have finished creating materials exactly when I planned to, but it required a lot more of effort to create high quality content. I estimated students will finish the whole course in 3 months. It turned out that there are a lot of topics covered by the program and to finish all homework students need 4-6 months.

We plan the second edition of the course to start in the beginning of 2022 – exactly end of January. Course is in Polish, but if there are people interested in the course they should subscribe to the notifications about the free webinar on 20th January 2022 at 20:00. You can also find more information about the course and webinar here.

Sandra: Could you also say a few more words about your company, Cufix? How many employees do you have? Do you have any plans for more initiatives and courses? Is there anything you could share with us? 🙂

Karol: At Cufix we work on computer vision related problems, and building 3D maps, based on 3D Lidar information. These are problems related to mobile robotics and self-driving cars. We work in a small team of 7 people. In our future plans, we aim to share a program focused on semantic segmentation and object detection. But for now I can’t tell any details.

Sandra: Do you have any book/s which have made a great impact on your career or life, and you could recommend? Doesn’t need to be data oriented but would be great definitely.

Karol: From the books I’ve read I can absolutely recommend one, which is Factfullness, a book really needed in current times. From it, you can learn about the current world situation, which is quite different from what we think. For example, the world population evolution or which countries are developed, or is this term still valid? I recommend this one to everyone, so am I this time.

Sandra: Any final words? Recommendations, advice for the future or call to actions? How readers can reach out to you?

Karol: Thank you so much for this interview! If you are interested in deep learning related topics in computer vision, feel invited to join the mailing group here. It was my pleasure, thanks!

My Reflections

What are your reflections after reading interview with Karol? For me this conversation was extremely inspiring. Let me explain why. His attitude is a perfect example that the sky is the limit if you are a true passionate of your job. Please do not get me wrong. I don’t mean to sell you any motivational tips or quotes that you are capable of doing anything. It’s all about hard work and understanding what really motivates you in life.

It’s visible from the first sight that Karol was into mobile robotics and engineering. He could stop with learning theory, find a stable job for many years, but he had chosen gaining experiences. He’s a true practicioner who knows how to best allocate potential. He has started from contributing to various projects and communities on his own and now focuses on sharing his knowledge and skills to others so that they could do same.

What I have also liked about this article is how Karol constantly emphasised the fact that nobody is capable of collecting all knowledge. You can be the top expert of the field, but with the amount of information generated this days, you cannot be up-to-date with everything and it’s completely okay. You need to find this maturity in yourself and learn how to select valuable information and invest your limited time. Should I learn a new skill on my own or should I choose a guided course? Even it’s more expensive, I will save on time spent on wandering unknown paths. Should I choose another university degree or contribute to the project? Karol’s experience perfectly shows how you can invest in yourself through years.

Summary

Interview with Karol was the second opportunity I had pleasure to accomplish due to my Instagram profile. Last year I have decided to broaden my network by adding this channel to my blog and it was a wonderful decision. First, I have managed to collaborate with Sylwana, one of the biggest AI activist in Polish community online (@programistkaAI). We have talk about her experiences as a woman in male-dominated industry here and now this amazing opportunity to deepen mobile robotics world. Now I’ve been able to deepen Karol’s experience with data science aspects I have had no chance to explore yet.

I am really grateful for today’s reality. The fact that we have the opportunity to meet people without leaving home, overcoming obstacles such as distance. It really makes me wonder if there are any inspiring personas within data science field you would like me to reach out this year. I have many ideas myself and I hope to surprise you more than once, but if I could fulfill any of your wishes, I am at your service.

See you in the next post!

Leave a Reply

Your email address will not be published.