Frank Pasquale – Black Box Society

There’s a repetition of this question when it comes to the transparency of our data being shared around, “should they tell us?”. With this rise of data sharing amongst big corporations as well as the government, we have created a world where it is organized by people who we can’t see but can see us. The book also poses a few critical strategies to “keeping the black boxes closed”. Real secrecy, legal secrecy, and obfuscation.

So going off of the question provided, how much of our information is being used should we be aware of? How would it affect our lives once we do know? Would it be easier to be blind to it?

Cathy O’Neill – The Era of Blind Faith in Big Data Must End

Algorithms built on the prospect of being a useful tool to determine success is a faulty approach. Algorithms are essentially just opinions embedded in code, and they reflect on our patterns and past practices as opposed to being the objective tool system that we had aimed it to be. As O’Neill put it, they automate the status quo. In order to bypass their silent and possibly detrimental actions, we have to check them for fairness. O’Neill lists them as having a data integrity check, audit the definition of success, consider accuracy, and recognize the long-term effects of algorithms.

Do you think that algorithms reflect on human biases? What other ways do you think that algorithms could potentially cause that could harm the lives of others?

Virginia Eubanks – Automating Inequality

Eubanks discusses the history and rise of the digital poorhouses and how it responds and recreates a narrative of austerity, the idea that there is not enough for everyone, so there’s this question of who deserves their human rights. She looks into the social services and how algorithms meant to aid the system to determine the people that are in need of help have actually caused a greater issue due to the history of how America treats the poor. She brings up an example where the system confuses parenting while poor as poor parenting, which exposes the system as a poverty profiling system that is a feedback loop of injustice.

She provides possible solutions to the issue, do you think that in the coming future we can somehow work together to override this issue and come to a better just future for the impoverished? Or do you think that this unfair system will continue to thrive for the benefit of the privileged and the detriment of the poor?

Janet Vertesi – My Experiment Opting Out of Big Data

I remember reading about this story in the past. In order to try and avoid getting bombarded with ads and suggestions by social media platforms regarding her pregnancy, Vertesi does her very best to essentially live an analog life. I commend her for the extent of work she has done to slide under the radar, however, it came with the cost of having tense relationships and the possibility of seeming like a criminal with the practices she was using.

Would you ever consider doing something like this to be under the radar of social media data collectors?

William and Lucas – The Computer Says No

I think that this is the perfect example of our reliance on technology. It is a simple yet effective execution of how we have been so trusting of our technology that we cannot see how it could cause an error, when in fact, just as humans, they can be just as flawed. There really isn’t a truly objective form of technology, as everything has been created to cater to our needs and have in some way shape or form, our tendencies to make mistakes as well. You can make a nearly perfect program, but biases from the creator can still be present, and we shouldn’t blindly believe everything the computer tells us just because we assume it is more reliable than humans.

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)

This chapter talks about how information becomes opaque, limited, and discreet, thus, narrowing our vision. The laws that govern these systems protect the commodity, but threaten our security. I think it’s interesting when the book talks about how our lives are public, but information is often not. I also recently signed a non-disclosure agreement and I certainly didn’t feel 100% comfortable.

Do you think companies should be transparent in exchange for more money?

Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)

This talk talks about how our faith in algorithms have become so great, that we become dependent on it. I can relate to this talk and how I’ve been judged based on an algorithm. As I apply to jobs, I’ve learned that many companies use ATS/ algorithms that track keywords and other information on resumes. I feel like this is unjust because it limits our vision on who people are into just data. She talks about how algorithms can also wreak havoc, but it takes time to do so.

Google’s search algorithm is what makes them money. Why do you think algorithms are considered so important?

Virginia Eubanks – Automating Inequality (talk, 45m)

This talk talks about how data/ technology becomes harmful for those who are less fortunate. She talks about cases where applications for medicare become neglected because of application errors. Those who need access to such resources the most, lose their benefits because of technological error. She talks about how these digital tools are always hiding besides us, and one error could lead to a spiral of other errors. I think that’s why the credit score system is a little freaky. I’ve been trying to build my credit since freshman year in college because this small data determines my house, my car, and my future.

How can we make data less personal?

Janet Vertesi – My Experiment Opting Out of Big Data…  (Time, short article)

I think this article and the experiment she carried out is very interesting. She tried to stay off the grid for 9 months during her pregnancy without trying to get caught by marketing/ advertisement. In order to do so, she withdrew cash and used a private internet server (thus talking about how it made her look like a criminal). I think it’s interesting to see how we’re monitored every day, and we can’t escape from it without compensating our comfort. Recently, I started using a sleep tracker app, and I was surprised that it could also figure out if I was snoring or not.

Can someone truly live off the grid?

Walliams and Lucas – The Computer Says No (comedy skit, 2m)

This skit demonstrates our dependency on technology and viewing it as an absolute truth. This also reminds me of the previous talks that we had to watch where people thought that google search results were all accurate, factual information. I think the problem displayed in this skit also displays the problem when it comes to data, big data, and data organization. We need to find a way to effectively store and search such data.

Do you think machines (such as kiosks at mcDonalds is more effective than humans?

• What are some of the techniques that these technologies use to trick us into forming a connection/relationship with a nonreal entity?

Maybe I could study the interactions I have between people and try to create that sort of attachment (nintendogs/visual novels)

Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)

****Cathy talks about data laundering, a process in which technologies hide ugly truths inside black box algorithms and call them objective. We are calling these things “meritocratic” when they are in fact non-transparent, and have the importance and power to wreak destruction. They are in-fact, if we think of one of the worst possible outcomes, mass-weapons of destruction. Except this is not “revolution, but evolution”. This adaptation of technology that integrates meta interfaces for data laundering is only revolutionizing and codifying biases that have been present in our society long before contemporary codes were able to solidify and exacerbate them.

Virginia Eubanks – Automating Inequality (talk, 45m)

****Virginia Eubanks talks about Automating Inequality as a creation of an insititution that upholds this “idea of austerity in which there is seemingly not enough resources for everybody, and that we have to make really hard choices about who deserves to attain those basic rights to get those resources”. She talks about the feedback loops of inequality that live in database storages and assume this idea of austerity, therefore looping it back into the system. Something very important she talks about is how the feedback loops create false negatives and false problems, which are the dangerous and debilitating parts of how this data can be used against individuals in favors of others. 

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)

—ABOUT BIG DATA and how there are multiple hidden layers of “firewalls” to get through, if you were to start calling out firms for being more transparent, for example, you would still have to get through all that incredibly hard to understand contract-jargon, that might end up being the discouragement needed to not press the subject further, “However, transparency may simply provoke complexity that is as effective at defeating understanding as real or legal secrecy”.

Transparency is not just an end in itself, but an interim step on the road to intelligibility.”

Janet Vertesi – My Experiment Opting Out of Big Data…  (Time, short article)

This experiment is based on a woman using using a site called Tor to source random servers for her searches. She did this because she wanted to evade the millions of data bots that would have invaded her online and in person existence through data referrals or advertisements. She used this site to bought everything in cash using gift cards to purchase things on amazon, and had to act like a criminal, using these gift cards via a site that is known for illicit activity.

This may seem insignificant, so what if companies send me coupons for baby stuff, but this exact same kind of “false negative”, it is, “—paved with the heartwarming rhetoric of openness, sharing and connectivity—actually undermines civic values and circumvents checks and balances”. 

Walliams and Lucas – The Computer Says No (comedy skit, 2m)

***I feel like this is something my parents would complain about, especially in hospital or administrative settings in which unnecessary bureaucracy takes places, and everything is handled through paperwork or computers, passed of by real hands, but only as a middleman. The idea of having a middle-person there for all these bereaucracy processes helps ease and alleviate the thought that these centers of “help” and importance (hospitals, DMVS, tax centers, banks, etc) are just databases that act as black boxes, storing our data as willingly or unwillingly as we give it. 

NOTES:

Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)

****Cathy talks about data laundering, a process in which technologies hide ugly truths inside black box algorithms and call them objective. We are calling these things “meritocratic” when they are in fact non-transparent, and have the importance and power to wreak destruction. They are in-fact, if we think of one of the worst possible outcomes, mass-weapons of destruction. Except this is not “revolution, but evolution”. This adaptation of technology that integrates meta interfaces for data laundering is only revolutionizing and codifying biases that have been present in our society long before contemporary codes were able to solidify and exacerbate them.  

Separates winners from losers

Or a good credit card offer

You choose the success of something

Algorithms are opinions embedded in code

Reflect our past patterns

Automate the status quo

That would work if we had a perfect world

But we don’t, we are reinforcing what we put in, our own collective memories and biases

They could be codifying sexism

Or any kind of bigotry

DATA LAUNDERING

Its a process by which:

1. Technologists hide ugly truths
2. Inside black box algorithms
3. And call them objective

Call them meritocratic

When they are secret, important, and destructive

Weapons of mass destruction

These are private companies

Building private algorithms 

For private ends

Eve the ones I talked about with police and teachers

Those are built by private institutions

And sold to the government 

This is called: Secret Sauce

PRIVATE POWER

They are profiting from wielding the authority of the inscrutable (impossible to understand or interpret)

Now you might think

Since all this stuff is private

Maybe the free market will solve this

There is a lot of money in unfairness

Also were not economic rational agents

Were all racist and bigoted in ways that we don’t understand and know

We can check them for fairness

Algorithms can be interrogated 

And they will tell us the truth every time

This is an algorithmic audit

1.Data integrity check

1. Definition of success; audit that

Who does this model fail?

What is the cost of that failure?

We need to think of: The long-term affects of algorithms, feed-back loops that are enacted.

Data scientists

We should not be the arbiters of truth

We should be interpreters of the voices…

__ VIRGINIA EUBANKS Automating Inequality

Virginia Eubanks – Automating Inequality (talk, 45m)

****Virginia Eubanks talks about Automating Inequality as a creation of an insititution that upholds this “idea of austerity in which there is seemingly not enough resources for everybody, and that we have to make really hard choices about who deserves to attain those basic rights to get those resources”. She talks about the feedback loops of inequality that live in database storages and assume this idea of austerity, therefore looping it back into the system. Something very important she talks about is how the feedback loops create false negatives and false problems, which are the dangerous and debilitating parts of how this data can be used against individuals in favors of others. 

We are creating an institution 

This idea of austerity

That there is not enough for everybody 

And that we have to make really hard decisions/choices about who deserves to attain their basic human rights

Tools of AUTOMATING INEQUALITY

More part of evolution 

evolution  than revolution

Their historical roots go back all the way to 120’s

1. Digital poorhouse assumes austerity (by assuming it it recreates it)
   1. DATAWAREHOUSES- private healthcare stores that information secretly, public health databases may not / do not
   2. If you can afford to pay your babysitter out of pocket, then that information about your family will not end up in the Data Warehouse

False positives problems 

seeing risk of harm where no harm actually exist

the system confusing system of parenting while poor, with poor parenting 

creating a system of Poverty profiling- spent so much time investigating and risk-rating families in their communities, created a feedback loop of i justice

that began with 

-families getting more data collected about them because they were interacting with county systems 

-having more interactions meant their score was higher

-having their score higher meant they were investigated more often, which means they were investigated more often

-and because they were investigated more data was collected on them and so forth so on the loop continues and data collection grows 

feedback loop- same as predictive policing 

FALSE NEGATIVES

Not seeing issues where issues might actually exist

Because there is barely any data on abuse in upper and middle class families in the data warehouse, the kinds of behaviors that might lead to abuse or neglect in those families can be ignored or misinterpreted by this algorithm because its not logged 

Geographically isolated places, or suburbs (misses key opportunities to outreach to places like these

Discrimination is occurring or getting initiated the most within the community, is when the COMMUNITY calls in to share those cases with the Child Welfare phone responders

-When call screeners receive these, there is also a bit of disproportion that happens in that moment

Referral – which is embedded in our cultural understandings of what a safe and healthy family looks like, and in the United States, that family looks white, heterosexual, and rich,

-Removing discretion from those online workers could remove a stop, to the massive amount of discrimination thats entering earlier in the process, and could potentially worsen inequality in the system rather than making it better

Discretion- energy

Never created or destroyed, its just moved

Who are we taking discretion away from, who are we giving it to?

-Removing discretion from frontline child welfare workers that make up a large amount of the diverse women work force

Giving it to the economists, computer engineers, and social scientists who are building the models

These tools, at their worst, can serve as an ~empathy override~

We’re allowing us to outsource to computers some of the most difficult problems and decisions we face as a society

Coordinated Entry System (used around the country, around the world)

Responds to the county’s extraordinary housing crisis

-works by assigning each unhoused person who they have managed to survey a number/score that falls on a spectrum of vulnerability

VISPIDAD

Vulnerability  Index and Service Prioritization Assistance Tool

-Serves those at top of scale, chronically homeless, 

-Serves those at the bottom of the scale, new homeless who need just little to help get back on their feet

Labeled as: not vulnerable enough to merit immediate assistance, BUT

Not STABLE enough to be served by the time-limited resources of rapid RE-housing

Leave people feeling as if they are included in a system that asks people to incriminate themselves in turn for a higher lottery number in the system

-Give folks your data and HOPE you get matched with a better housing opportunity

-or close yourselves out of most housing resources in the community at all

Data management from survey of the organization is shared with a 161 organizations, and because of federal law and databases, one of those orgs is the LAPD (homeless management information system)

___BLACK BOX SOCIETY

There is even an emerging fi eld of “agnotology” that studies the “structural production of ignorance, its diverse causes and conformations, whether brought about by neglect, forgetfulness, myopia, extinction, secrecy, or suppression.” 

But what if the “knowledge problem” is not an intrinsic aspect of the market, but rather is deliberately encouraged by certain businesses? What if fi nanciers keep their doings opaque on purpose, precisely to avoid or to confound regulation? That would imply something very different about the merits of deregulation. The challenge of the “knowledge problem” is just one example of a general truth: What we do and don’t know about the social (as opposed to the natural) world is not inherent in its nature, but is itself a function of social constructs. Much of what we can fi nd out about companies, governments, or even one another, is governed by law. ****

Laws of privacy, trade secrecy, the so- called Freedom of Information Act— all set limits to inquiry. They rule certain investigations out of the question before they can even begin. We need to ask: To whose benefi t?

Some of these laws are crucial to a decent society. No one wants to live in a world where the boss can tape our bathroom breaks. But the laws of information protect much more than personal privacy. They allow pharmaceutical fi rms to hide the dangers of a new drug behind veils of trade secrecy and banks to obscure tax liabilities behind shell corporations. And they are much too valuable to their benefi ciaries to be relinquished readily.

Even our po liti cal and legal systems, the spaces of our common life that are supposed to be the most open and transparent, are being colonized by the logic of secrecy. The executive branch has been lobbying ever more forcefully for the right to enact and enforce “secret law” in its pursuit of the “war on terror,” and voters contend in an electoral arena fl ooded with “dark money”— dollars whose donors, and whose infl uence, will be disclosed only after the election, if at all.6

But while powerful businesses, fi nancial institutions, and government agencies hide their actions behind nondisclosure agreements, “proprietary methods,” and gag rules, our own lives are increasingly open books. Everything we do online is recorded; the only questions left are to whom the data will be available, and for how long. 

Knowledge is power. To scrutinize others while avoiding scrutiny oneself is one of the most important forms of power.8 Firms seek out intimate details of potential customers’ and employees’ lives, but give regulators as little information as they possibly can about their own statistics and procedures.

Sometimes secrecy is warranted. We don’t want terrorists to be able to evade detection because they know exactly what Homeland Security agents are looking out for.12 But when every move we make is subject to inspection by entities whose procedures and personnel are exempt from even remotely similar treatment, the promise of democracy and free markets rings hollow. Secrecy is approaching critical mass, and we are in the dark about crucial decisions. Greater openness is imperative. (No transparency!)

Financial institutions exert direct power over us, deciding the terms of credit and debt. Yet they too shroud key deals in impenetrable layers of complexity. In 2008, when secret goings- on in the money world provoked a crisis of trust that brought the banking system to the brink of collapse, the Federal Reserve intervened to stabilize things— and kept key terms of those interventions secret as well. Journalists didn’t uncover the massive scope of its interventions until late 2011.13 That was well after landmark fi nancial reform legislation had been debated and passed—without informed input from the electorate— and then watered down by the same corporate titans whom the Fed had just had to bail out.

Deconstructing the black boxes of Big Data isn’t easy. Even if they were willing to expose their methods to the public, the modern Internet and banking sectors pose tough challenges to our understanding of those methods. The conclusions they come to— about the productivity of employees, or the relevance of websites, or the attractiveness of investments— are determined by complex formulas devised by legions of engineers and guarded by a phalanx of lawyers.

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)

—ABOUT BIG DATA and how there are multiple hidden layers of “firewalls” to get through, if you were to start calling out firms for being more transparent, for example, you would still have to get through all that incredibly hard to understand contract-jargon, that might end up being the discouragement needed to not press the subject further, “However, transparency may simply provoke complexity that is as effective at defeating understanding as real or legal secrecy”.

Transparency is not just an end in itself, but an interim step on the road to intelligibility.”

***So why does this all matter? It matters because authority is increasingly expressed algorithmically.22 Decisions that used to be based on human refl ection are now made automatically. 

Software encodes thousands of rules and instructions computed in a fraction of a second. Such automated pro cesses have long guided our planes, run the physical backbone of the Internet, and interpreted our GPSes. In short, they improve the quality of our daily lives in ways both noticeable and not

The same goes for status updates on Facebook, trending topics on Twitter, and even network management practices at telephone and cable companies. All these are protected by laws of secrecy and technologies of obfuscation.***

Though this book is primarily about the private sector, I have called it The Black Box Society (rather than The Black Box Economy) because the distinction between state and market is fading****CAPITALISM IS TURNING INTO GOVERN. PROXYS

We are increasingly ruled by what former politi cal insider Jeff Connaughton called “The Blob,” a shadowy network of actors who mobilize money and media for private gain, whether acting offi cially on behalf of business or of government.24 In one policy area (or industry) after another, these insiders decide the distribution of society’s benefi ts (like low- interest credit or secure employment) and burdens (like audits, wiretaps, and precarity).

But a market- state increasingly dedicated to the advantages of speed and stealth crowds out even the most basic efforts to make these choices fairer. 

Obfuscation involves deliberate attempts at concealment when secrecy has been compromised. For example, a fi rm might respond to a request for information by delivering 30 million pages of documents, forcing its investigator to waste time looking for a needle in a haystack.17 And 

the end result of both types of secrecy, and obfuscation, is opacity, my blanket term for remediable incomprehensibility.18

However, transparency may simply provoke complexity that is as effective at defeating understanding as real or legal secrecy. 

 

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)

This book talked a lot about the secrecy of the government, banks, and software  while relating it all into the same realm. I found it interesting how Frank related the secrecy and volatility of banks years after the recession to the data firms stealing your data as we speak. One of my favorite quotes of this reading that summed up some of these points concisely was “All these are protected by laws of secrecy and technologies of obfuscation.” I think this summed up a lot of what Frank mentioned about confusing the consumer and being very opaque. He briefly discusses the long prospectuses, and disclosure agreements of corporations and how their intent is to just confuse the readers. He brings up how some are tens of thousands of pages, and they often reference to other things, that reference to other things. I think one thing you can take away from this reading is that we as consumers and members of society should really think about our right to have access to our data being collected, and/or the knowledge how it is collected.

Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)

This was in the final 30 seconds of the Ted Talk but was a very powerful quote.. “Data Scientists should not be arbiters of truth, we should be translators of ethical discussions that happen in larger society.” I thought this summed up this talk very well. There is a big issue with algorithms making decisions in certain fields that are biased, and are not making logical decisions. The fact that teachers are not only gettin scored, but losing their jobs because of a score does not make sense if they are liked by everyone they work with and work for. I thought the Fox news scandal was interesting to bring up. It’s ironic that after the entire scandal, Fox acted like they were going to clean slate and remove all these previous biases and favoritism, yet they make hiring algorithms based on all their previous data. 

Virginia Eubanks – Automating Inequality (talk, 45m)

Eubanks builds a digital poorhouse based on the economic depression in 1820. 

It’s not really fair the the medicaid workers were replaced by algorithms in Indiana and then 1 million cases were denied for so called not completing the online application. This puts somebodies life at stake because of not signing a form online.

Janet Vertesi – My Experiment Opting Out of Big Data…  (Time, short article)

Even though this article is short, and tells me a lot of information that I have already been told, it really did still surprise me and have a powerful theme. I knew it would be difficult if somebody tried to go completely anonymous online and avoid every single form of data tracking, but I didn’t know you would look like a criminal doing this. “But avoiding the big-data dragnet meant that I not only looked like a rude family member or an inconsiderate friend, but I also looked like a bad citizen.” This is crazy how these are the implications of literally trying to buy something without having a data footprint. Now this really is the best way to explain to somebody how publicized our society is now, and how much surveillance capitalism has taken over our world.

Walliams and Lucas – The Computer Says No (comedy skit, 2m)

Hilarious. I love how he asked for the survey at the end. It’s very true that sometimes you’re talking to these receptionists and it’s like they just talk straight out of the computer. Delegitimatizing what you say sometimes because the computer says something else. Super funny skit though, watched it twice. 

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)

• I really enjoyed the metaphor used in the very beginning. That the light is the powerful source that he has to work around. Comparing that kind of power to big data. The fact that nobody knows too much about big data is the reason why it is so powerful. This article argues that this “knowledge problem” is probably for good reason, “to whose benefit”. For pharmaceuticals, they mention that they are allowed to hide the dangers of a drug. Everything we do online is recorded, our credit, our phone location, so where does that information go? For how long? By whom?
• How can we get that control back? Do we need to continue to rely on whistleblowers? Will companies ever do anything when they know we’re eating at the palm of their hand?

Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)

• Algorithm = data + definition of success. She mentions how they take the teachers and scores of their students into an algorithm to shame them, yet nobody outside could access the algorithm. Now how is that fair, when the algorithms could be inherently biased or does not have the full context of information? I also thought the idea of an algorithm of fox news would filter out women because they have not really been the face of success in the past, or most likely people of color as well. This is the type of systemic racism that is put in place today, with police. She suggests algorithm checking through a) data integrity b) definition of success c) accuracy and d) long term effects.
• How can we allow the public to gain access to these algorithms?

Virginia Eubanks – Automating Inequality (talk, 45m)

• It’s interesting how throughout the years, even though we change the algorithm, we still end up being racially biased. First it was containment (1819), then investigation (1873), and then digital surveillance -> prediction (1973) which we still use today to keep POC, specifically black people, in their place. Because even with good intentions, they can still have bad outcomes.
• Have algorithms ever prevented white men from doing anything?

Janet Vertesi – My Experiment Opting Out of Big Data…  (Time, short article)

• This kind of experiment is really interesting to see just how long you can keep a secret from big data. The internet, other people’s comments, pictures, credit cards, cookies, phone tracking, messages, etc. What I thought was a really powerful comment was the idea that “No one should have to act like a criminal just to have some privacy from marketers and tech giants”. We shouldn’t be required to give our personal information in order to not be perceived as a criminal.
• Will big data just get even bigger? Is there no way to minimize it?

Walliams and Lucas – The Computer Says No (comedy skit, 2m)

• The reason that this can even become a comedy skit is the fact that this kind of thing is so relatable and happens so often. We don’t question the system even when the system is clearly wrong. Instead of fact-checking it, we’re more likely to just (as Cathy O’Neill put it) put our blind faith into it. Yes, this is just a sketch, but life imitates art, and there is a sense of accuracy to what is being portrayed.
• How can we convince the general public to be more cautious in their trust?

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)
It’s interesting to me how sometimes a business is considered a legal person, and sometimes it’s not. It appears that this distinction is simply based on laws that were arranged to benefit large corporations the most. Businesses have more privacy rights than people do, which is even more absurd when businesses use the lack of consumer privacy as a means of business.

Our data is being mined to benefit corporations. We know this, yet we still happily hand over our data every day. Is it possible for us to minimize or eliminate the amount of data that is collected on us?


Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)
In a nutshell, algorithms are made by people, and people are faulty. We have biases (even with good intentions), and we make mistakes. To blindly accept an algorithm’s data as the prime and only truth is only dragging us down. We can still use data and have positive effects, so long as we keep in mind that algorithms are tools, and tools sometimes break.

What kinds of data do we blindly accept in our everyday lives?


Virginia Eubanks – Automating Inequality (talk, 45m)
Data plays a big role in how a company views success, but the data isn’t always correct. In addition, the algorithms which retrieve the data are often made by people who have no real-world experience with what they’re trying to get data on.

How could we measure success without using data?


Janet Vertesi – My Experiment Opting Out of Big Data…  (Time, short article)
As my oldest is starting to work, the ideas of banks and credit cards are becoming a topic of discussion around my house. This article made me rethink my spending habits, and I wonder if I could successfully, and comfortably, go “off grid” with my finances and purchases.

One thing in the article that stuck out to me was the aspect of other people. I’ve been battling my feelings for privacy and my feelings of narcissism for a long time. I’ve been thinking a lot about being tracked, not just by Google, but also by, well, anyone in the world, really. I still remember a huge part of my life where I could essentially remain anonymous. It was lovely. Now, things are just… weird. I’ve “deleted” my social media accounts on multiple occasions, most recently for over a year (I don’t remember exactly how long, but it was a while). A lot of the time, my “friends” just wouldn’t respect my wishes of not having pictures put up before I approve them, and not to tag me in anything before I approve it. I had various reasons for these requests, but, like the author, some people just didn’t get it or didn’t care.

While advertisements are aimed to make companies money, could it also be that our information could be used to design products that we would actually use that would help improve our lives?


Walliams and Lucas – The Computer Says No (comedy skit, 2m)
This short skit highlights how some people take data as the truth over what a living human being says. The little girl was in to get her tonsils removed, but the computer data said she was there for a double hip replacement. She quite obviously doesn’t need a hip replacement, and the viewers are left laughing at the receptionist’s inability to see real-world facts when the computer data contradicted it. While this is laughable, it’s actually happening all around us. I wouldn’t be surprised to find out that I, too, have fallen victim to the “data is the truth” trap.

We’re painting data inconsistencies or slight nuances in a bad light. While precise and correct data would be necessary for certain things (administering medicine, for example), I wonder if it’s not such a bad thing to have a larger margin of error in regards to less life-threatening aspects. If you’re using a fitBit, does it really matter if your number of steps is, let’s say even 1000 steps off? You’re up and getting active, and in my eyes, the fitBit data is more like a game anyway. It wouldn’t matter if a virtual character really walked 1543 steps or not because the game could encourage you to problem solve, thus working your brain, which is better than being sedentary most of the time.

Frank Pasquale – Black Box Society – chapter 1 (pp 1-11)

When we discuss obfuscation in our seminar it is always from our end to the big corporations. Thinking about it from the other end is even more scary given the fact that we may never be able to pull back the curtains on what is actually happening. Our attempts of obfuscation still occur on their territory, we think we are giving the middle finger to the man in the ivory tower, but in reality we are probably just a small anomaly in a huge sea of information. We already know the public atrocities many big businesses conduct and whenever the back end deals are exposed through things such as the panama papers they are big for the moment and then nothing ever happens.
Cathy O’Neill – The era of blind faith in big data must end (Ted Talk, 13m)

Algorithms have the connotation of efficiency. Big corporations definitely love endorsing the idea of algorithms being the future. O’Neill talks a lot about the idea of how we define success in terms of an algorithm and algorithms capability to be wrong. Formulas for math equations are taught because if used correctly will get your answers every time, algorithms are being taught in the same manner even when it applies to human work ethics. People who develop the algorithms go off of a standard that is totally foreign to their own experiences. Its scary to think about how people who have zero experience with what we are doing are defining how to do it “optimally”.
Virginia Eubanks – Automating Inequality (talk, 45m)

We have talked a lot about the idea of how systems, digital or not, can fundamentally be biased. Algorithms are not systems, it immediately makes me think of Hans Haake’s work as his work such as condensation cube demonstrates a system that is out of human direct manipulation. Condensation isn’t biased and it doesn’t care who you are or your background. People love to teach others that algorithms function in the same way. That its all behind a “machine learning program” that somehow that means that machine is not biased.
Janet Vertesi – My Experiment Opting Out of Big Data…  (Time, short article)

Vertesi’s experiment to hide her own pregnancy was something I did read about before. I think about all the ways that she has to take extra steps to just function at the status quo. It’s always so dumbfounding when people brag about crimes on social media or even admitting to crimes via text messages because they think they are isolated within a group of people that they know.
Walliams and Lucas – The Computer Says No (comedy skit, 2m)

I know this is talking about how computers take our information and they want correct and true answers. But, the think that I am thinking about the most is the increasingly pathological driven our society is. The skit feels so unrealistic only because when people are sick they will undoubtedly google every thing they have and they will give any details that they think sites will need.

Black Box Society

I would like to say I have never heard the story of the man and the lamppost just to put it out there. But it is a really good metaphor for our relationship with technology and the futility of security we have now. What are some ways we experience what the author defines as agnotology? How does it effect our daily lives? I think at its core this article is about knowledge and how a lot of our knowledge is hidden from us by the society around us for a variety of reasons, some malicious, like using our data for profit, others not so much. But all effect how we go through the world and how we can access and deal with data and personal privacy.

The Era of Blind Faith in Big Data Must End

The fact that teachers are getting fired because an algorithm said so reminds me of the surveillance articles about students being tracked by apps that were faulty. It puts technology first and literal humans second and its so easy for it to be broken or otherwise abused its crazy. What O’Neil said is really on point, algorithms repeat patterns, and this would be fine if our world was perfect but it isn’t. To blindly rely on them is only furthering disparities, because just like other data and AI related things, they are made by humans, and humans have bias.

Automating Inequality

My experiment opting out of big data

A side note before I even start, I find the term sociologist of technology to be interested when typically sociology is the study of human society itself. I assume this means like looking at society in terms of technology but the phrasing was interesting to me.

I’m surprised this experiment was actually even nominally successful considering the examples the author had of friends and family members still going against her wishes with things like Facebook messages. This article really shows how its almost (read probably) impossible to be completely off the big data grid, and that it is impossible to simply opt out and have that be enough. There isn’t really a choice as the author states, and its not a matter of simply leave it if you dont like it, because our society is set up to make it impossible to leave.

The Computer says No

I always say comedy is the most truthful. This one is difficult to respond to because its so short but it is very quick to the point of our reliance on technology. It reminds me of the times when I used to do compliance checks for stores that sold tobacco and the cashiers wouldn’t trust the computers if they thought I was old enough (or purposefully didn’t type it in right.) Its blindly following whatever the computer says regardless of what your thoughts on the matter might be.

Frank Pasquale’s Black Box Society: I get paranoid thinking about how the government/financial institutions want to remain secretive while still having the ability to track/surveil the general public. As an American, I know all too well about our government lying or not telling the whole truth about our past wars; and I’m sure they would prefer the public to not know anything at all. I also know that Super PACs (i.e. CEOs of corporations) can fund ads -with no limit- for or against political campaigns which must have a huge algorithmic impact. Honestly, I can’t hear about ‘nondisclosure agreements’ without instantly thinking of Mike Bloomberg and Donald Trump. Pasquale uses the word ‘black box’ to refer to how we are constantly being recorded, but in a secretive manner by a secretive entity. What is described in this book is very reminiscent to me of Cambridge Analytica and surveillance capitalism (coined by Zuboff). I think this short clip from The Simpsons Movie (which came out before Snowden) humorously depicts the ‘black box’ society we live in. Although I know about obfuscation, I didn’t know about real v. legal secrecy (one refers to locking a door or making a password, the other is about keeping sensitive information a secret i.e. not giving out SS#s). I guess the Equifax data breach would be a great example of how legal secrecy can be violated.

• Is transparency the answer to uncovering government/financial/corporate institutions’ secrets?
• If so, how might we obtain more transparency?
• If not, is there another way? Should we just live with the unknown secrets?

Cathy O’Neill’s The era of blind faith in big data must end: O’Neill first asks, “What if algorithms are wrong?” and then follows up with, “To make an algorithm you need data and a definition of success.” According to O’Neill, everyone uses an algorithm with or without code. One example of an algorithm used without code has to do with the process of making a meal. Whoever is making a meal has to factor in the time, resources, and energy required to make it (data). Also, the success could be eating enough vegetables; but if there are children, eating enough sugar could be their own success (which probably isn’t good for them in the long run). Ultimately algorithms are opinions, not objective science. It’s sad that teachers lost their jobs over inconsistent data. It seems hard enough just to be a grade school teacher in the U.S. …Codifying sexism, racism, homophobia is definitely a major concern of mine. I think the game Portal is a great example of how algorithms can go wrong. The main robot in Portal, GLaDOS, practically embodies an algorithm running tests (in the name of science) which ultimately achieve the death of people.

• What are other examples of algorithms that don’t operate on code? -Could getting a degree/diploma be considered another example? (With data being the classes/hours it takes and success being graduating)
• What kind of opinions do you think social media algorithms have?

Virginia Eubanks’ Automating Inequality: Eubanks starts the lecture by stating that we are building a digital version of a 1820s poorhouse. Apparently in the 1820s there was an economic depression and of course the concern was of people being ‘dependent’ on the government and not of people living in poverty. Unfortunately it has been two centuries later and this same sentiment seems to be around today. I didn’t know ‘county farm’ is coded language for where a poorhouse once was… Anyway, Eubanks gives a couple of modern day examples of this phenomenon. In Indiana, applying for food stamps (a.k.a. SNAP) turned into a digital process which ended up leaving a lot of eligible people without the program. Occasionally those who went through the process of signing up would get notified of an error, but there were no specifics as to what the error was. The story of a woman with the last name Young was heartbreaking! She couldn’t go through the welfare process while in the hospital with cancer, so of course she loses her coverage and dies before the court case reaches a verdict. According to Eubanks, the middle class can pay with private benefits so there is not as much data on them. A system that only has data from the lower class is obviously going to conflate parents living in poverty with poor parenting. For child protective services this lack of data from middle class families can result in false negatives (i.e. not seeing harm when there may be some) and for lower class families this can result in false positives (i.e. seeing harm when there is none). Not to mention that there is a racial bias when it comes to child welfare. This racial bias is most apparent when communities call on black/biracial families (what Ruha Benjamin referred to when talking about the citizen app and the BBQ Becky). Eubanks says that California’s VI SPDAT collects data on the homeless for housing opportunities, but it’s possible for the LAPD to abuse that data. Also a guy who goes by Uncle Gary living in skid row filled out the VI SPDAT multiple times, but was not seen as vulnerable enough for housing. It just goes to show that welfare systems are not better helped by status-quo algorithms. I think this lecture had a stronger connection to The Computer Says No skit since healthcare is brought up. I can’t imagine a 5 year old needing a hip replacement, tonsils sounds more like it. I swear there’s too many customer service surveys these days…

Janet Vertesi’s My Experiment Opting Out of Big Data…  Opting out of personal data collection isn’t easy when it makes you appear as a rude family member, inconsiderate friend, and/or a bad citizen. Vertesi talks about her experience trying to not have big data detect her pregnancy… I can’t imagine downloading something probably used for the dark web (Tor) just to visit BabyCenter.com in privacy or setting up a separate Amazon account and buying gift cards in cash to purchase things online. I feel bad for the uncle who PMed her a congratulations, but yeah, I see Facebook as trying to calm the “Big Brother is watching you” anxiety by calling it a ‘private’ message. I swear I’ve seen ads on my Facebook Messenger too! I have a friend who’s very protective of their privacy so I only message them through Signal. They also use duckduckgo and Firefox. I should probably get into that habit, but I don’t think I can give up my gmail and google drive.

• Do you think it’s even possible to live a life completely ‘off-the-grid’ today? Why or why not?
• How many conveniences/relationships would you have to give up in order to do so?