WhyLabs

What do you like best about WhyLabs?

The core functionality is explicitly important when you are having AI/ML solutions running on production. Instead of spending time on manual checks (that can be prone to error) you can set up monitors to inform you if something is going wrong. It is flexible enough to build your cusotm metric and monitors which allows for versatile usage.

What is highly important to me is the contact with WhyLabs team. Their support is superb which results in fact that even if something is difficult to implement, I know I have people who will help me get the best out of the platform.

What do you dislike about WhyLabs?

Building custom monitors and metrics is a chellanging experience. I struggle sometimes with code issues so better FAQ or documentation could be helpfull to avoid them.

What problems is WhyLabs solving and how is that benefiting you?

We are having multiple user facing projects that are working on production. We need to know if their performance is deteriorating, if we see some bias, data drift or missing data.

What do you like best about WhyLabs?

One of the standout features of WhyLabs is its robust data observability capabilities. It provides continuous monitoring of data pipelines and ML models, allowing teams to quickly identify issues like data drift, model degradation, and training-serving skew. The platform's privacy-preserving integration ensures that data can be analyzed without moving or duplicating it, which is critical for maintaining security and privacy in sensitive industries like healthcare and finance​

What do you dislike about WhyLabs?

One potential drawback of WhyLabs is its relatively limited user reviews and feedback due to its newness in the market, making it harder for potential users to gauge its real-world performance at scale. This lack of detailed reviews can raise concerns about its maturity and support infrastructure​.Additionally, since it’s a newer platform, some advanced features might still be in development, and there could be steep learning curves for teams unfamiliar with observability tools in machine learning​.

What problems is WhyLabs solving and how is that benefiting you?

Data quality issues: It helps detect and address data drift and data integrity problems early, which is crucial for maintaining accurate and reliable ML models​

What do you like best about WhyLabs?

* The customer support is very helpful and proactive

* Tool allows for easy ingestion of big number of features and setting up initial monitoring on them

* We can use it to monitor both: input quality and model performance

* The alerts can be raised to specific group of users via specific channels (email/slack), which is helpful

What do you dislike about WhyLabs?

* It can be challenging to setup the monitoring in the correct way when it comes to sensitivty - it requires a lot of trial and error

* Some actions are not possible via UI and require specific API calls

* Documentation can be hard to navigate

What problems is WhyLabs solving and how is that benefiting you?

Monitoring model performance and input data quality in one place.

What do you like best about WhyLabs?

Great to collaborate with; very responsive; really appreciate their OHs to help out with issues that pop up; many out-of-the-box solutions for different kinds of ML models which really helped us out given the wide variety of ML models we run at the company.

What do you dislike about WhyLabs?

Nothing major to mention! We got everything resolved and the team was very helpful.

What problems is WhyLabs solving and how is that benefiting you?

Data Drift and ML Monitoring

What do you like best about WhyLabs?

The foremost aspect that stood out for us was the ease of integration. It was quite straightforward, almost as same as the sample code to put LangKit into our LLM pipeline and upload it to WhyLabs. The getting-started docs were clear and concise. Even for companies with limited resources like ours, WhyLabs AI Observatory proves to be an accessible and easily adaptable solution.

What do you dislike about WhyLabs?

The data visualization tools are highly capable, although there is some learning curve to master it. However, their customer success and support team are responsive, helpful, and have consistently gone above and beyond to ensure that we utilize the platform to its full potential.

What problems is WhyLabs solving and how is that benefiting you?

The monitoring capabilities of WhyLabs AI Observatory are the main reason why we use it. We have very limited control of external LLM services. WhyLabs provides alerts and insights into those LLM models, enabling us to identify and address potential issues promptly.