"Several"
This past week, my team at InMoment has been doing some analysis for a client that runs a chain of opticians. We wondered if we could use LLM instructions to work through feedback left by customers one-by-one to determine whether they mentioned a timeframe in their feedback1. If someone left a comment like:
My 2 new pairs of glasses were ready to pick up in less than 2 weeks! Great job!
...could we programatically get that '2 weeks' value and turn it into a 14 day integer?
If we could, we could then do a nice comparison of time versus satisfaction score.
But we found a problem. Humans aren't computers and their conception of time is fuzzy. We found plenty of comments like:
Had an annual eye examination and was told my eyesight had gotten worse so I needed a new pair of glasses. When they arrived several weeks later...
In fact, look at the first example again. The person didn't specify that their glasses were ready in precisely two weeks. They were ready in less than two weeks.
We built out a bit of a ruleset, which is as follows:
- If an exact timeframe is mentioned without any qualifier, use that number of days precisely
- "Less than" seems to mean "closer to this value than the next lowest example", e.g. "less than two weeks" means "closer to two weeks than one week", so around eleven days
- "More than" can be treated in the same way: "closer to this value than the next highest example"
- The unit being expressed is important. "Less than a month" will almost certainly mean more than two weeks, likely somewhere around the three week mark or so. "Less than two months" will mean more than a month, but likely somewhere around the six week mark
- "Several" is problematic
Hoo boy, "several" is the worst.
By the book several's literal meaning is "more than one" but by common consent in the UK it means "more than two". Because it looks like "seven" there's a temptation to conflate the two, but "seven" and "several" turn out to have completely different origins and the similarity is a pure coincidence2. "Several" changes depending on the unit being expressed - the number of units in "several days" is likely very different to the number of units in "several weeks" or "several months" or "several years".
It's also likely that what is meant by "several" in one region or market is totally different in another.
Thankfully in our dataset the word didn't come up that often3, but each time there was a debate among the team as to what number it sort of represents. We went with the following:
- "Several days" is less than a week, likely around four or five days
- "Several weeks" is less than a month. If it was two weeks it's more likely that a customer would say two weeks, so it's likely somewhere around three weeks, likely a little more
- "Several months" is anything more than two months but less than half a year
What we ended up doing is reporting the results back with fairly large intervals: here are the results for the people reporting a timeframe of one-to seven days, here's the interval for those reporting eight to 14 days, etc. With the data we had and the calls we needed to make, that worked pretty well.
But it's not great data science, is it?
If we had more time and access to the CRM data, we'd have made a more informed decision: done a census of the data where customers had reported waiting "several weeks" and seeing if the record gives a precise number, used some logic to determine an average. We could also have run a recontact campaign, asked those customers for a bit more detail and some specifics.
But sometimes you have to make the pragmatic choice; it's likely our fuzzy intervals would have resulted in the same result regardless.
Oh, and in case you're interested: if people mention a timeframe they're broadly happy within two weeks. After that, it isn't pretty.