text is a temporary lossy workaround

as an extremely text-biased person with an unjustifiable fetish of long-form text, it pains me to have to start to accept that text (hence writing and reading) is not the most effective method for information transfer. text might be the most efficient (storage and transmission wise) and relatively convenient (skimmable, indexable, tangible), but I am…

our phones didn’t destroy our attention span

perhaps this “social media and smartphones have ruined our attention span” issue is really just FOMO on steroid and less an actual neurological change? do you actually have bad attention span or you are just tired, in a rush, and constantly in FOMO? so a combination of 1) a lack of mental bandwidth, 2) a…

[30hbc] Paying for conversations and miscellaneous

idk what it is about conversations that allow them to draw insight out of you with ease. Looking at prompts and exercises in workbooks, I don’t feel motivated to work on them. It’s different than if I imagined someone asking me those Qs. The latter feels more effortless and engaging. I guess this is what…

What are people trying to do with data? JTBD and DMCA

Why do people want data? What do people use it for? What are they trying to do, really? Put simply, four verbs: Decide, Monitor, Create, and Automate. Let’s first look at something called the JTBD framework. JB.. what? Jobs-to-be-done. Here’s a good description: https://www.productplan.com/glossary/jobs-to-be-done-framework/ The jobs-to-be-done framework is an approach to developing products based on…

Different levels of readiness when embarking on web data projects

I know I said we will look at different ways people use web data this week, but I realised there’s one other thing related to the six challenges worth discussing before moving on. It is the different levels of readiness / maturity when people embark on web data projects. I call it: “data-needs journey”. By…

Scaling: The Third Challenge in Web Data Acquisition Project

Sometimes you find that your current setup doesn’t give you the sufficient data to achieve your business goals. You might feel the need to increase the frequency, add the breadth of the data collection parameters for example keywords, zipcodes, categories, or add new data sources. You need to scale the project. What do you need…

6 Challenges in Web Data Acquisition Projects

Let’s talk about the six common challenges I see people face in data acquisition projects. Setup Banning Scaling Maintaining Cost Quality (of service and/or of data) Before diving in, the definition and scope of “data acquisition project” here is when the data you need exists on a website / other HTTP-accessible resources. This type of…