Random Thoughts – Randocity!

This article seeks to show that how Nielsen Media Research chooses its ratings families may alter the accuracy of the Nielsen’s ratings. More than this, this article seeks to uncover just how antiquated and unreliable Nielsen’s household rating system actually is. Let’s explore.

What is Nielsen?

I’ll give a small synopsis here, but Wikipedia does a much better job at describing who and what Nielsen Media Research (one of this company’s many names) is. For all intents and purposes, I will refer to Nielsen Media Research as simply Nielsen for the purpose of this article.

Nielsen is a research group who seeks to identify how viewers, among other avenues of information that they gather, watch Television. During the 70s, this was the primary means by which TV executives learned the ratings fate of their television programs.

How does Nielsen work?

Nielsen still relies on its Nielsen households to provide the vast majority of its television ratings information. It does this by sending out unsolicited mail to households around the country attempting to solicit a household into becoming a Nielsen household. By using this moniker, it means the family who resides at a specific household must do certain things to not only participate in the Nielsen program, but must also provide feedback to Nielsen around its viewing habits.

How does Nielsen collect its ratings information?

According to Nielsen’s own site, it says the following:

To measure TV audiences and derive our viewing metrics (i.e., ratings, reach, frequency), we use proprietary electronic measuring devices and software to capture what content, network or station viewers are watching on each TV and digital devices in the homes of our Nielsen Families. In total, we measure hundreds of networks, hundreds of stations, thousands of programs and millions of viewers. In the U.S., electronic measuring devices and millions of cable/satellite boxes are used to provide local market-level viewing behaviors, enabling the media marketplace to gain a granular view of TV audiences.

What that means is that, as a Nielsen household, they will send you a device and/or require you to install certain software on your existing devices which will “measure” your viewing habits. In other words, they spy on what you’re watching and it reports back to Nielsen what you specifically watched and for how long. For example, Nielsen might install software onto your smart TV device, Roku, TiVO, Apple TV or possibly even your cable TV provider’s supplied box.

Nielsen may even be willing to supply you with their own device, which you will place in-line with your existing TV and devices. It does say “devices and software”, meaning one or both can be used.

Rural vs Urban

Typically, larger urban city areas tend to vote Democrat more often than Republican. These urban areas are also typically more densely populated. On the flip side, rural areas tend to vote Republican more often than Democrat. Why is this information important? It’s important to understand these facts because it can drastically alter the accuracy of Nielsen’s ratings. Let’s understand why.

For participating in being a Nielsen household, you’re given a stipend. In other words, you’re paid for this service. Let’s understand more about this pay. You’re paid around $10 a month to participate. If you remain a Nielsen household for a certain period, around 6 months, Nielsen will pay you a bonus. All told, for 6 months of service, a Nielsen household will receive around $200.

Here’s where the Urban vs Rural comes into play. Rural areas tend to be more depressed economically. Meaning, income is generally less and the need for extra money is, therefore, higher. Urban areas tend to boom more economically meaning the need for extra money is, therefore, lessened.

If a rural household receives a card inviting them to become part of the Nielsen family, explaining all of the “benefits” (including the pay), rural viewers are much more likely to take Nielsen up on their pitch. It seems easy enough to get paid simply for watching TV. On the other hand, urban areas are less likely to take Nielsen up on their offer not only because the pay is so low, but because urban viewers are much more savvy around their privacy.

Who would intentionally invite a company into your household to spy on you, even for money? One might say, well there’s Alexa. Alexa offers benefits to the user far greater than what Nielsen provides. Nielsen provides spying for cash. Alexa offers app features, smart house features, music, calling features, recipe helpers, and the list goes on. Nielsen’s device(s) and software(s) don’t provide those much extended features.

Nielsen’s spying is one tracked and only helps out TV executives. I might add that those TV executives PAY Nielsen to gain access to this information. Which means that if you’re a Nielsen household, you’re getting paid out of money collected from TV executives. In effect, it is the TV executives who effectively sign your Nielsen paycheck that you receive. I digress.

Random Solicitation

Make no mistake, Nielsen solicits households through a random mail selection process. It sends pitch cards out to inform and solicit households to participate. They may even include a crisp $1 bill to entice the household. Nielsen knows that a certain percentage of people will take Nielsen up on their offer to participate in the program.

The difficulty is that this selection process relies on random chance for whomever chooses to participate. This goes back to Urban vs Rural argument. Because depressed areas are more “hard up” for cash, they are more likely to take Nielsen up on their offer than Urban areas, who urban viewers are not only likely to be mistrustful of spying using digital devices, these people also don’t necessarily need the small-ish amount of cash that Nielsen is offering… considering the amount of time required to watch TV (and do whatever else Nielsen requires). Yes, Nielsen requires you to watch TV to participate. The whole thing doesn’t work unless you actually watch TV.

This ultimately means that it is more likely rural Republican areas of the country are over represented in Nielsen’s households and equally likely Democrat areas to be under-represented in Nielsen’s ratings. While Nielsen has no control over who chooses to accept the “Nielsen Household” solicitation, Nielsen does control the parameters to entice people into the program. Thus, their parameters are skewed toward lower income households, which are likely to be in predominantly rural areas.

In other words, depressed rural areas are far more likely to need the extra cash and be willing to jump through Nielsen’s hoops than more affluent urban areas. That’s not to say that there won’t be a percentage of viewers in urban areas as some households in those areas may elect to participate.

Disposable Income

Urban areas can be a bit more affluent than rural areas. Urban area residents may have more in disposable income, but also because it’s a larger city, it has more entertainment options. This means entertainment options besides watching TV. When you live in a small rural town, entertainment options can be extremely limited even if disposable income is available. Rural townships tend to encourage more TV watching more often than urban areas where night clubs, restaurants, theme parks, opera, live theater events, shopping and large cinemas are common. More entertainment options means less need to watch TV as often.. except for specific shows.

Thus, urban viewers are less likely to want to participate in Nielsen’s household program than rural viewers, whose entertainment options may be limited by both what’s available near them and by their disposable income.

Extrapolation

Here’s the crux of Nielsen’s problems. Based on the over and under represented areas due to Nielsen’s flawed selection process, they attempt to make up for this by extrapolating data. Regardless of how the households may be skewed, Nielsen intends to extrapolate its data anyway.

Nielsen estimates that it has around 42,000 households participating in 2022. Though, I’d venture to guess that that number is not completely accurate. I’d suggest Nielsen may have perhaps half that number actively participating at any one time. There might be 42,000 households signed up as a Nielsen household, but likely only a fraction actively participate at any specific moment in time.

For example, not every household will watch a specific sporting event when it’s on. Only those who truly enjoy watching a specific football game might be watching a specific game. This could drop that 42,000 households down to under 5,000 viewers. If it’s a local sporting event, it could drop that number down well below 1,000 and maybe even below 200 actively watching.

200 equals 1 million, 5 million, 100 million?

How does this affect the ratings? Good question. Only Nielsen really knows. The problem is, as I stated above, Nielsen uses extrapolation.

What is extrapolation? Extrapolation is the process of using 1 viewer to represent many viewers. How many is a matter of debate. It is a process that Nielsen has employed for many years, and it is highly inaccurate. It makes the assumption that for every one viewer watching, there will be a specific number also watching. How many are extrapolated is really up to Nielsen. Nielsen must come up with those numbers and herein lies the inaccuracy.

Effectively, Nielsen fudges the numbers to appear great (or poor) depending on how it decides to cull the number together. In other words, extrapolation is an exceedingly poor and inaccurate way to determine actual viewership numbers. Yet, here we are.

Digital Media Streaming

With digital streaming services, such as Netflix, Hulu, Amazon and Crackle… more specifically, devices such as DVRs like TiVO and devices like Apple TV, Nielsen’s numbers may be somewhat more accurate when using these devices. However, one thing is certain. Nielsen still doesn’t have 100% accuracy because it doesn’t have 100% of every TV household participating.

Again, Nielsen’s numbers may be somewhat more accurate because we now have active digital streaming devices, but Nielsen still employs extrapolation to inflate the data they collect. Nielsen takes the numbers they collect, then guess at how many might be watching based on each single viewer’s behaviors.

Why Extrapolation over Interpolation?

Interpolation requires two distinct sets of data points in which to fill in the interior data gap between those two sets. Filling in data between two distinct sets of data is a bit more accurate than attempting to guess at data points outside of them.

With viewership numbers, it’s only one set of data at a single point in time. Everything that is gleaned from that single set of data is always considered “outside” or “extrapolated” data. There’s nothing in a single data set to interpolate. You have 42,000 households. You have a smaller number watching a TV program at any point in time. That’s all there is.

If Nielsen ran two unique and separate sets of 42,000 households of viewers (a total of 84,000 viewers), interpolation would be possible between those to separate sets of 42,000. Nielsen doesn’t utilize this technique, thus making interpolation of its collected data is impossible.

How Accurate is Extrapolation?

Not very. I’ll point to this StackExchange article to explain the details as to exactly why. In short, the larger the number gets outside of the original sample size, the larger the margin for error… to the point where the error outweighs the value of the extrapolation.

One answer provides this quote:

[Extrapolation] is a theoretical result, at least for linear regression. Indeed, if one computes the so-called ”prediction error” (see this link, slide 11), one can easily see that the further the independent variable 𝑥 is away from the sample average 𝑥¯ (and for extrapolation one may be far away), the larger the prediction error. In the link that I referred to one can also see that in a graphical way.

In a system where there is no other option, such as during the 70s when computers were room-sized devices, extrapolation may have been the only choice. Today, with palm sized internet enabled phones containing compute power orders of magnitudes faster than many of those 70s room-size computers, continuing to use extrapolation honestly makes zero sense… especially when accuracy is exceedingly important and, indeed, required.

Extrapolation Examples

If 1 Nielsen viewer represents 1,000 viewers extrapolated (1:1,000), then 100 Nielsen households watching suggests 100,000 viewers may actually be watching. If 100 Nielsen viewers watch a program and each household represents 100,000 viewers (1:100,000), then this suggests 10,000,000 viewers may be watching. Just by changing the ratio, Nielsen can alter how many it suggests may be watching. Highly inaccurate and completely beholden to Nielsen making up these ratios. As stated above, the larger the number diverges from the original sample size, the larger the margin of error… possibly making this data worthless.

These suggested extrapolated viewership numbers do not actually mean that that many viewers were, in reality, watching. In fact, the real viewership number may be far, far lower than the extrapolated numbers suggest. This is why extrapolation is a bad, bad practice. Extrapolation is always error prone and usually in the wrong way. It makes too many assumptions that are more than likely to be wrong.

Unless the person doing the extrapolation has additional data points which logically suggest a specific ratio is at play, then it’s all “best guess” and “worst error”.

How many businesses would choose run their corporation on “best guess”? Yet, that’s exactly what TV executives are doing when they “rely” (and I use this term loosely) on Nielsen.

Biased

Even above the fact that extrapolation has no real place in business, because of its highly inaccurate and “best guess” nature, these numbers can be highly biased. Why? Because of the Urban vs Rural acceptance rates.

Unless Nielsen explicitly goes out of their way to take the under vs over represented nature of Nielsen households into account when extrapolating, what Nielsen suggests is even more inaccurate than I even suggest just from the use of extrapolation alone.

CNN vs Fox News

CNN has tended to be a more liberal and, thus, a Democrat favorable news organization. Though, I’d say CNN tends to be more moderate in its liberal Democrat leanings. Fox News, on the other hand, makes no bones about their viewpoint. Fox News is quite far right and Republican in too much of its of leanings. Fox News is not always as far right as, for example, Alex Jones or other extremist right media. However, some of its leanings can be as far right as some quite far right media. Here’s an image from the Pew Research Center that visually explains what I’m describing:

Whether Pew’s research and datapoints are spot on, I’ll leave that for you to decide. I’ve reviewed this chart and believe it to be mostly accurate in terms of each outlet’s political leanings. Though, I personally have found PBS to be somewhat closer to the “Average Respondent” location than this chart purports… which is why even Pew might not have this chart 100% correct. For the purpose of CNN, Fox News and Hannity, I’ve found this chart to be spot on.

As you can see in the chart above, Fox News itself is considered a right leaning news organization, but not far off of center at around a 2. However, the Sean Hannity show is considered just as far right as Breitbart at about 6-7. CNN is considered slightly left leaning at around a 1 (less left leaning than Fox News is right leaning at 2).

What does all this mean for Nielsen? It means that those who are Republican, which tends to include more rural viewers than urban, those rural viewers tend to be conservative. Because Nielsen is more likely to see participation from rural viewers than urban viewers, due to its enticement practices, this skews Nielsen’s accuracy towards conservative viewership and away from liberal viewership. Nielsen’s enticement practice isn’t the only problem which can lead to this skew, though.

Meaning, Fox News viewership numbers as stated by Nielsen may be highly overestimated and inaccurate. Quantifying that more specifically, Fox News viewers may be over-represented where CNN viewers may be severely under-represented. It further means that unless Nielsen actually realizes this liberal vs conservative under vs over representation disparity in its Nielsen households (respectively) and, thus, alters its extrapolated numbers accordingly, then its viewership numbers published for CNN vs Fox News are highly suspect and are likely to be highly inaccurate.

Worse, Fox News is owned by Rupert Murdoch. Because this man is in it for the cash that he can milk from the Fox News network, he’s more than willing to pay-for-play. Meaning, if he can get companies to favor Fox News by asking them for favors in exchange for money, he (or one of his underlings) will do it. Murdoch can then make more money because more advertisers will flock to Fox News under the guise of more viewership. Fake viewership is most definitely lucrative. Because Nielsen extrapolates data, this makes faking data extremely easy.

Unlike YouTube where Google has no reason to lie about its reported views, Fox News has every reason to lie about its viewership, particularly if it can game other companies into complying with its wishes.

Nielsen Itself

Nielsen purports to offer objective data. Yet, we know that businesses are helmed by fallible human CEOs who have their own viewpoints and political leanings and who are in it for the money. One only needs to look at Rupert Murdoch and Fox News to understand this problem. Some CEOs also choose to micromanage their company’s products. Meaning, if Nielsen’s current CEO is micromanaging its ratings product, which is also likely to be Nielsen’s highest moneymaking product, then it’s entirely possible that the ratings being reported are biased, particularly in light of the above about Rupert Murdoch (who is also a Republican).

Conflict of Interest

When money gets involved, common sense goes out the window. What I mean by this statement is that since TV executives / networks pay Nielsen to receive its ratings results periodically, Nielsen is beholden to its customers. The word “beholden” can have many meanings in this “sales” context. Typically in business, “beholden” means the more you pay, the more you get. In the case of Nielsen, it’s possible that paying more to Nielsen potentially means that business may get more / better ratings. That sort of breaks the “objective” context of Nielsen’s data service. It’s called “Conflict of Interest”.

In essence, in this case it could represent a pay-for-play solution, a true conflict of interest. There’s honestly no way to know what deals Nielsen has brokered with its clients, or more specifically with Rupert Murdoch’s Fox News Network. Most companies who do sales deals keep those details close to the vest and under non-disclosure binding contracts. The only way these deals ever get exposed is during court trials when those contracts can become discovery evidence for a trial. Otherwise, they remain locked in digital filing cabinets between both parties. Even then, such contracts are very unlikely to contain words disclosing any “back room” verbal handshake deals discussed. Those deal details will be documented in a separate system or set of systems describing how to handle that customer’s account.

Let me count the ways

There are many problems in the Nielsen’s rating services that may lead to highly inaccurate information being released. Let’s explore them:

1. Nielsen’s solicitation of households can easily lead to bias due to its probability of luring in people who are hard up for cash (e.g., rural Republicans) vs those who are not (e.g., urban Democrats).
2. Nielsen’s products and software spy on knowing users about viewership habits. Spying of any variety is usually viewed with skepticism and disdain, especially these days and especially by certain types of people in the population (usually liberal leaning individuals). Rural Republicans are less likely to understand the ramifications of this spying (and more willing to accept it) than urban Democrats (who tend to be more likely to work in tech based businesses and who see this type of spying as too intrusive).
3. Nielsen’s numbers are “fortified” using extrapolation. Fortified is a nice way of saying “padded”. By padding their numbers, Nielsen staff can basically gyrate the numbers any way they want and make any channels viewership numbers look any particular way. Which ties directly into…
4. Nielsen sells its ratings product to TV producers and networks. Because these deals are brokered separately for varying amounts of money, the network who pays the most is likely to see the best results (i.e., pay-for-play).
5. Nielsen moved away from its “on paper” auditing system to the use of digital device auditing. Because Nielsen removed the human factor from this ratings equation (and fired people as a result), it also means that fewer and fewer people can see the numbers to know what they truly are (or at least were before the extrapolation). Fewer people seeing the numbers means higher chances of fabrication.

Looking at all of these above, it’s easy to see how Nielsen’s numbers could be seriously inaccurate, possibly even intentionally. I won’t go so far as to say, fake, although that’s entirely possible.  However, because Nielsen employs extrapolation, it would be easy for a Nielsen staffer (or even Nielsen’s very CEO) to make up anything they want and justify it based on its “proprietary” extrapolation techniques. Meaning, numbers stated for any network’s viewership could be entirely fabricated by Nielsen, possibly even at a network’s request or possibly even as part of that network’s deal with Nielsen.

In fact, fabrication is possible based entirely on number 4 above. A TV network could pay significantly to make sure their network and their programming is always rated the highest, at least until they stop paying for it. With Nielsen’s extrapolation system and when data can get played fast and loose, it’s entirely possible for such a sales scenario to manifest.

Why are Nielsen’s Numbers Important?

Advertising. That’s the #1 reason. Companies using TV advertising wish to invest their advertising dollars into channels with the highest viewership. The higher, the better. Nielsen’s ratings are, therefore, indicative that a higher ratings share means higher viewership. The problem is, Nielsen’s extrapolation gets in the way of that. Regardless of whether or not cheating or fabrication is involved, the sheer fact that extrapolation is used should be considered a problem.

The only thing Nielsen really knows is that of the 42,000 Nielsen households that it has devices in, only a fraction of those households watched a given program or channel at any specific time. Meaning, the real numbers of viewership from Nielsen offers a maximum of 42,000 viewers at any moment in time… no where close to the millions that they claim. Any number higher than 42,000 is always considered fabricated whether extrapolation or any other means is used to inflate that number.

That companies like Procter and Gamble rely on those 42,000 Nielsen households to determine whether to invest perhaps millions of dollars in advertising on a channel is suspect. That companies have been doing this since the 70s is a much bigger problem.

In the 70s, when there was no other way to really determine TV viewership, Nielsen’s system may have held some measure of value, even though it used extrapolation. However, in 2022 with live always-on internet enabled phone, tablet, computer, game console and other smart TV devices, measuring actual live viewers seems quite feasible directly from each device tuned in. If someone is live streaming CNN over the Internet, for example, it’s not hard to determine and count this at all. If hundreds of people are streaming, that should be easy to count. If millions, it’s also easy. Why extrapolate when you can use real numbers?

The days of extrapolation should have long ended, replaced by live viewer tallies from various digital streaming devices, such as phones, computers and Apple TVs. Whether these devices are allowed to phone home to provide that data, that’s on each viewer to decide. If the viewer wishes to opt-in to allowing their viewership metrics to be shared with each TV station, then that’s far more realistic viewership numbers than Nielsen’s extrapolated numbers. If they opt-out, then those stations can’t see the numbers. Opting in and out should be the choice of the viewer.

That’s where privacy meets data sharing. Some people simply don’t want any of their private data to be shared with companies… and that’s okay. That then means some level of extrapolation (there’s that word again) must be used to attempt inflate the numbers accordingly.

Let’s consider that 42,000 is 0.01273% of 330 million. That’s trying to represent the entire population of TV viewers in the United States from less than 0.01% of people watching. Insane! With always-on digital devices, if 10% opt out, that’s still provides 90% more accurate viewership numbers than relying on Nielsen’s tiny number of households. Which means there’s way less amount of data to attempt to extrapolate. That advertisers don’t get this point is really surprising.

Auditing

You might think, “Well, isn’t Nielsen audited?”

Most companies dealing with numbers are typically audited. Unfortunately, I’ve found that working in a tech business which sees regular audits can still have fabrication. How? Because those who work on the technical side of the house are not those who get audited. Meaning, those systems administrators who maintain the logs and records (i.e. databases) aren’t under the scrutiny that the financial side of the house gets.

If it relates to money and sales, auditing of the accounting books is a regular occurrence and must uphold specific standards due to legal requirements. Auditing when it relates to anything else is catch-as-catch-can, particularly when laws don’t exist. Meaning, the auditors must rely on the statements of staffers to be accurate. There’s no way for an auditor to know if something has or hasn’t been fabricated when viewing a log.

Worse, if the company employs a proprietary algorithm (read private) to manage its day to day operations, auditors typically are unable to break through its proprietary nature to understand if there’s a problem afoot. In other words, auditors must take what’s told to them at face value. This is why auditing is and can be a highly inaccurate profession. I should also point out that auditing isn’t really intended to uncover treachery and deception. It’s intended to document what a company states about specific questions, whether true or false. Treachery and deception may fall out of an audit, but usually only if legal action is brought against the company.

In the case of money, it’s easy to audit records of both the company and third parties to ensure the numbers match. In the case of proprietary data, there’s no such records to perform this sort of matching. What an auditor sees is what they must accept as genuine. The only real way that such deception and fabrication becomes known is if an employee performing such fabrication blows the whistle. An independent auditor likely won’t be able to find it without a whistleblower. Because jobs tend to be “on the line” around such matters, employees are usually told what they can and cannot say to an auditor by their boss. Meaning, the boss might be acutely aware of the fabrication and may instruct their employees not to talk about it, even if directly asked.

In fact, employees performing such fabrication of data may intentionally be shielded from audits, instead throwing employees who have no knowledge at the auditors. It’s called, plausible deniability.

Overall

None of the above is intended to state that Nielsen fabricates numbers maliciously. However, know that extrapolation of data is actually the art of data fabrication. It takes lower numbers and then applies some measure of logic and reasoning that “makes sense” to deduce a larger number. For example, if one person complains of a problem, it’s guaranteed a number of other people have also encountered the same exact problem, but didn’t complain.

The art is in deducing how many didn’t complain. That’s extrapolation by using logic and reasoning to deduce the larger number. Extrapolation clearly isn’t without errors. Everyone who deals in extrapolation knows there’s a margin of error, which might be as high as 10% or possibly higher and which grows as the extrapolation data size increases.

Are Nielsen’s ratings numbers accurate? Not when you’re talking about 42,000 households attempting to represent the around 122 million households with TVs. This data doesn’t even include digital phones, tablets and computers which are capable of streaming TV… which smartphones alone account for about 7.26 billion devices. Yes, billion. In the United States, the number of smart phone owners is around 301 million. There are more smart phones in existence in the United States (and the rest of the world) than there are TV’s in people’s homes.

So, exactly why does Nielsen continue to cling to its extremely outdated business model? Worse, why do advertisers still rely on it? 🤷‍♂️

↩︎