In June 2023, I spent three weeks deep down in the shrimp industry in Andhra Pradesh. This is not my first attempt in this, I have done a shorter deep dive in January 2023 where I travelled with a friend into middle of nowhere to go to a processing plant, we ended up getting stuck at the end of the evening and had to hitch hike our way back to civilization, But that is for another story. I wanted to learn more about this shrimp domain since this is the field I will need to capture in my models at Kontali. This region produces more than 80% of the 4.7 million tons of shrimp India exported in the last 5 years. Here’s what I found on the ground.
Initial Objective (For the analysts) #
The mission was straightforward:
- Get a pulse of how things are at the ground in the Indian market at that moment.
- Get access to WhatsApp groups and contacts.
Good to have objectives (For myself) #
- To get a better understanding of the industry.
- Get access to new “data”.
- Get an understanding of data collection methodologies.
- Look at Data science value generation scenarios.
Value for shrimp industry people and why would they talk to me? #
I had to think about what I could offer:
- Expertise and trust in the shrimp and seafood industry.
- That we possess a very large collection of seafood data and models.
- Convince them that we are a Neutral entity.
- Best practices in anonymising what they share with us.
- Maybe Kontali Edge product is interesting (but I am not sales).
Meetings on the ground over 3 weeks in June 2023 #
Here’s who I met:
- 8 Farmers
- 5 Brokers/middlemen
- 3 Processing companies
- 3 Hatcheries
- 2 Startups
- 2 Testing labs
- 1 Feed producer
- 1 chemicals importing firm
- 1 YouTube educator
What I Got Out of It #
- Access to more than 10 WhatsApp groups, both local and international.
- Established valuable connections within the industry.
- Obtained tables backed by a substantial sample size, for our models.
- Gained market intelligence on the current conditions.
- Lot of the “real” information is spoken in native language, never written and on somebody’s local machine. Continuous contacts and people on ground matter.
The Harsh Reality #
But here’s what really hit me:
- There is a prevalent reluctance to invest in quality data, resulting in a low willingness to pay for such data products. We would need better marketing to make them see otherwise.
- Many believe they already possess the best available data and if we share our methods, they want to create it themselves.
- According to the industry experts, Overproduction has dimmed prospects for price improvements and better industry conditions.
- Data-driven methodologies are actively employed to micromanage workers and low prices from farmers.
Discussion Points #
This whole experience left me with some big questions:
- Is it worth the time to make this deep dive?
- How to do this in better way?
- How do we deal with constantly changing data source? (non-etl-able data)
- How do you deal with information if they write down stuff in local language but with English letters?
What I Learned #
The biggest takeaway? Most of the valuable intel never makes it to any database. It’s all WhatsApp conversations in Telugu written with English letters, voice notes, and conversations over tea. The guys who really know what’s happening are veterans who’ve been doing this for decades, and they’re not about to start filling out data entry forms.
The processing companies have all the fancy dashboards and analytics, but they’re using them to squeeze farmers even more. Meanwhile, everyone thinks they have the best data already, so why would they pay us for ours?
Three weeks taught me that this industry runs on relationships, not spreadsheets. If you want real data, you need real people on the ground, constantly. There’s no API for trust.
Final Thoughts #
Would I do it again? Probably. The shrimp domain is messy, informal, and resistant to traditional data science approaches. But that’s exactly what makes it interesting.
Slides from a presentation at a workshop in Oslo (October 2023): #
