#Prefect vs airflow software#

This guy is definitely not shipping models to production. But, on the other hand, analysis paralysis isn’t much better (back to point 2). Finally, you realize that this field is fast-changing, so committing to a vendor that might be obsolete, or not support the formats/tools you’ll use tomorrow is problematic.

#Prefect vs airflow software#

You understand that you’re part of a team – your company is going to be a rocket ship, and you’ll have a huge team soon enough (if you don’t already have one), and you already need to work with other teams in your organization, like domain experts, software engineers, and DevOps.

Moving fast is of the essence since you need to show the rest of your organization that this new AI thing can actually translate to business value, and you want to focus on what you do best – understanding your data, transforming it, and either translating it into business insights, or training an ML-powered component for your product/company.

Since they each have a learning curve, it’s scary to try another one, potentially wasting more time on something you’ll end up needing to build internally. Worse yet, you’ve tried a few of these tools and have been disappointed. You have a lot of issues, it seems like there are a lot of tools for them, but everyone is claiming they can do everything, and it’s hard to know what tool to use for each task.If you’re a data scientist working on machine learning projects, you might have noticed a recurring issue in your work. I provide a list of common problems and OSS solutions for those problems, with comments on when they’re better/worse for that issue. It’s also a good idea to think about these challenges from a problem-solution perspective, as opposed to a “give me the awesomest tools” approach. It’s best to use open-source, best-of-breed, modular solutions. Data scientists have challenges and need tools to overcome them.