Knowledge Matters™

This week the Department of Homeland Security received the results from a study by the National Research Council that the DHS funded. The study's conclusion: data mining for terrorists does not work and it invades the privacy of innocent citizens. Searching for the terrorist needle in the haystack of phone-calls and emails is counter-productive.

I told you so... here, here, here, here, and here.

However, if you start with a known suspect or two, you can roll up the rest of the network with normal surveillance techniques. Rather than dig through millions of records looking for that unknown terrorist pattern, give the phone company the numbers of known suspects/terroists and have them return the 2-step network neighborhood of each number -- now you can see the terror suspect's extended social graph. This approach with social network analysis also uncloaks street gangs and criminal networks.

UPDATE: ABC News has more on surveillance of US Citizens abroad.

This week, I am blogging on one of the Amazon blogs -- Omnivoracious.

Multiple people are posting about various approaches on how book purchases may influence the upcoming 2008 US presidential election. I am grateful to be included with such experts as John Zogby, Bill Bishop, and Andrew Gelman. Each of us has a different approach for how to mine Amazon data for interesting insights.

Enjoy!

How many NOTs in your network? [NOT = Non-Obvious Tie]

You probably can't answer that, because the connections are...
n o n - o b v i o u s .

Ties/links/connections/relationships that are not obvious to me may be obvious to someone else, and vice versa. Unfortunately, the knowledge of those ties may not be as valuable to those those who know, than to those who do not know.

Confused? Let me share a few stories...

This afternoon I was looking through the access logs to my business web site. I saw some interesting NOTs. Two *.mil orgnaizations were visiting certain pages. Would my friends at Booz-Allen-Hamilton would find that information of great value? BAH sells products and services to DoD clients. The BAH rainmakers would love to know what military branch X and Y are interested in and what search terms they used, and what pages they spent a long time on.

Then I noticed visitors from the Dark Web [web sites that support/idolize Jihad]. I wondered... wouldn't the military folks, that visited just an hour before, like to see what the jihadi supporters were looking at, and what referrer pages and search terms they used?

Finally a third NOT, all in one quick browse of the logs. Wouldn't the Oracle business intelligence folks like to know what SAP employees were looking at, and vice versa?

My web site is not that different from thousands of other business web sites that are browsed each day by a diverse mix of concurrent visitors. If only visitor X knew that visitor Y was there and what they were looking at. Maybe a few days later visitor Z arrives and produces the same clickstream as visitor X -- what does that say about the two visitors?

Our choices reveal who we are. Our search paths give insight into who we are not... yet.

Even though though we don't know who each of these visitors are [the actual people representing their organization], their behavior reveals much about what is important in their organization and what they are trying to learn. Just like the networks of Amazon purchase data reveal interesting political patterns without revealing who the actual purchasers are, these "choice & search" networks reveal much about the organizations our individually unknown visitors are from.

Finally, a NOT story with a happy ending.

A blogger friend of mine was being stalked by someone who took great umbrage to one of the posts on the blog. The stalker assumed a Jane Doe identity and started to leave increasingly abusive comments. By checking the blog logs my friend saw the IP address of the abuser and saw that they were spending a lot of time on the site. They were waiting for the blogger to answer one of the abusive comments or for a friend of the blogger to leave a positive comment that they could pounce on.

The stalker was using his employer's network. Yes, she was a he, a quick domain look up revealed what company he worked for. The company sold financial products and services to consumers. My friend thought they would guard their reputation vigorously. My friend contacted their media relations department and explained the situation and provided them the IP address from which the comments were coming. Soon the company's director of IT Security called and explained how seriously they were taking this misuse of their network. Within a day, the IP address and the abuser's fake Jane Doe GMail address were tracked to a PC within a company branch office. Both HR and I/T approached the abuser and basically read him the riot act. He quickly admitted his guilt and pleaded for forgiveness. No details, other than the above, were provided by the company. As a former HR person, I suspect he will be very closely watched and most likely terminated at the next display of similar behavior. His opportunities for promotion and advancement are now severely limited. Dumb guy.

A similar process, to the above, is currently being used to trace the hacker who broke into Sarah Palin's YahooMail account(s). Many people think they are anonymous and invisible on the internet. Not so -- NOTs will quickly reveal what you are after, where you came from and finally, who you are.

We have seen how obesity and smoking are affected by the network around an individual. Alas, it also works for crime and terror. From the BBC report on how people become terrorists:

The picture that emerges is of largely intelligent people finding direction in the networks of associates they keep.

"The work on pathways into terrorism indicates that it comes out of a social process; it comes out of a series of contacts that terrorists have with other individuals," Professor Canter told BBC News.

"At the broader level, everything has to be done to undermine the idea that individuals think of themselves solely in terms of any particular group of sub-group - be that fundamental Muslims or supporters of a football club . Once people only think of themselves in those terms, then that sets the seeds for conflict."

Re-read that last paragraph above.

Any group that isolates itself, by just focusing on itself, sows the seeds of conflict with the rest of society. Groups isolated from the rest of society are not automatically terrorists [look at the Amish], but they are places where confirmation [what we all think is correct] and conformation [you must not think differently than us] are oppressive forces acting on group members.

We confirm, you conform are the unspoken rules in any isolated group, whether they be criminals, terrorists, religious zealots or those overly fanatical about their political party or sports team.

Below are some of the close internal ties of the 9/11 hijackers. Included in their group were others who had already carried out acts of terror -- setting an example of behavior for the rest.

I'm surprised it took Amazon this long to exploit their own political book buying data. The Amazon maps go to where the power is -- state by state -- the Electoral College.

The Amazon maps indicate whether each state's residents purchased a larger number of red or blue books. Comparing my recent book network maps to the above pictured book volume map, seems to show that while the Left read a larger variety of books, in most states the Right buys greater quantities of a smaller set of books.

Some interesting questions remain...

• are book buyers influential with voters in their social network?
  
• how many voters does a typical political book reader reach/influence?
  
• are Amazon book buyers representative of the political book buying population?
  

Amazon clusters the political book sales data into 60 day slices. When you go back earlier in the year 2008, you see more blue or neutral shaded states. The most recent 60 day slice of 2008 [before and after the conventions] shows a blossoming of deep red. If polemics predict [no one knows for sure], and the election were today, then the latest Amazon political book data maps appear to indicate a President McCain in 2009.

Another nice feature that Amazon provides is to go back to 2004 and look at the political book buying patterns then. The most interesting change of patterns is between the map just before the 2004 election and the map immediately after the 2004 election.

During the Republican National Convention, John McCain's acceptance speech indicated that he was running against the current state of affairs in Washington DC. Will he be different than the current administration of George Bush? Is McCain a Republican of a different stripe? Will a McCain administration be different than a Bush administration? With only opinions and no data, we can argue all day and all night.

Maybe some data can help us see behind the rhetoric? One way to gain insight into possible future behavior is to look at who is donating to the campaign and hoping to influence a new administration.

I downloaded data of the top bundlers of donations for the 2000 and 2004 Bush campaigns and the 2008 McCain campaign. What's the overlap of donors between the Bush and McCain campaigns? Will the same people influence both campaigns/administrations? Or will it be starkly different groups? Or something in between?

From The Fix @ washingtonpost.com...

McCain's bundler program is built on the incredibly successful "Pioneer" and "Ranger" program built by George W. Bush in 2000 and perfected four years later. The most valuable individuals in a bundling program are not necessarily the wealthiest individuals but rather those individuals with the deepest Rolodexes and a willingness to ask and ask and ask.

Below is a map of those who donated to BOTH Bush and McCain. The campaigns are shown as the two red nodes on the left of the map. The green links show donations coming into the McCain 2008 campaign. The blue lines show donations coming into the Bush campaigns of 2000 and 2004. The 128 bundlers, who have contributed to both McCain and Bush, are shown in the arc on the right.



Most of McCain's 534 large bundled donations [76%] came from donors who did not donate to either of the Bush campaigns. Yet, this kernel of 128 bundlers keeps consistency across all three Republican campaigns in the 21st century.

Although the majority of McCain fundraisers were not Bush fundraisers, the Gang of 128 may not allow McCain to wander too far from the current philosophy and approach. If elected, McCain may be different than Bush, but he might not be that different.

Update: McCain 2008 Bundler List & Obama 2008 Bundler List

When you know the net you can quickly get to the information or resources you need in your local community.

So, the statements below about John McCain's vetting process for his VP candidate are puzzling. Did they not know how to scroll through the network via key access nodes [a.k.a. network weavers] or did they just not do it?

From the New York Times...

"They didn't speak to anyone in the Legislature, they didn't speak to anyone in the business community,"said Lyda Green, the state Senate president who lives in Wasilla, where Palin served as mayor.

Representative Gail Phillips, a Republican and former speaker of the state House, said the widespread surprise in Alaska when Palin was named to the ticket made her wonder how intensively the McCain campaign had vetted her.

"I started calling around and asking, and I have not been able to find one person that was called," Phillips said. "I called 30 to 40 people, political leaders, business leaders, community leaders. Not one of them had heard. Alaska is a very small community, we know people all over, but I haven't found anybody who was asked anything."

The current mayor of Wasilla, Dianne M. Keller, said she had not heard of any efforts to look into Ms. Palin's background. And Randy Ruedrich, the state Republican Party chairman, said he knew nothing of any vetting that had been conducted.

State Sen. Hollis French, a Democrat who is directing the ethics investigation, said that no one asked him about the allegations. "I heard not a word, not a single contact," he said.


In Athens, Ohio, one of the key community access nodes is June Holley -- she can probably connect you to any part of the community or economy, either directly, or in one or two introductions/steps. June is not the only community access node in Athens -- there are dozens. You don't have to find the best one -- many well connected nodes will work as a productive starting point in your journey through the net.

The people quoted above all seem to be key members of the Alaskan state government -- all probably within 2 steps of each other, and network neighbors of anyone you would want to talk to when checking references and reputations.

Was the vetting rushed, or did they really not know the net, and how to get the key information they needed?

How can 30-40 key political players not know what is going on?

Sounds like WMD 2.0 to me. Others call it toadyism.

What do you think?

UPDATE: They only checked within their own little clique -- those with broad reach to many cliques/clusters/groups were not contacted. Selective use of intelligence? Flawed patterns repeated?

From the Anchorage Daily News...

Thomas Van Flein, the Anchorage lawyer representing Palin and her office in the legislature's investigation into the firing of former Public Safety Commissioner Walt Monegan, said he spoke to several representatives from McCain before Palin's selection was announced Friday.

But Van Flein appears to be in a small minority in the vetting of Palin.

On Sunday, The Washington Post quoted McCain campaign manager Rick Davis as saying the FBI conducted a background check of Palin.

But Monday, the FBI told the Atlantic Monthly no such check took place.

If the newspapers can find key nodes in the net in just a few days, why couldn't the McCain campaign?

When a member of the echo chamber inquires within the echo chamber all s/he hears are the known echoes. And since the echoes appear to come from several places at once s/he believes that everyone thinks so, and therefore it must be the truth.

It is amazing how many of our current problems come down to the realization that it's the network, the connectivity, that matters. In most situations we know how to fix and enhance the nodes in the network. The links, and their patterns and structure, are the hard problem.

We are making progress in alternative energy production, but we still fail at energy distribution. Windmills and solar energy collectors have made great progress -- we just can't get the energy from where the wind blows and the sun shines to where the great population centers are. To do that requires a well-designed power distribution grid. Many critics of the current grid describe it as "third world" in design, quality and capability. Today's New York Times describes the distribution problem well.



Above is a network map of a portion of the US electric grid. Life is great if you live in one of the densely connected clusters using electricity generated nearby. Things start to get real complicated if energy needs to transferred from one cluster to another cluster in grid. Distance destroys. Electricity does not flow like information or water or oil. It is not easy to direct, and much electricity is lost to heat when transferred over long distances. On the internet, 100 packets sent from Cleveland all arrive in New York wholly intact -- not so with a 100 MW of electricity generated in Cleveland and sold to NY. Even more electricity would be lost going to Miami, and forget about LA. It makes no sense to transfer electricity made in Cleveland to Los Angeles -- most of it would be lost during the trip.

Not only does physics get in the way, so do local interests. Then you have another power problem -- that of political power. Doing a social network analysis of the electric grid quickly points out key nodes and links that are highly between transfer points on the grid. They become gatekeepers/bottlenecks and either extract a toll for the transfer, or refuse transfer and require the buyer and seller to find a longer route to get from point of generation to point of consumption. And remember -- distance destroys.

Energy independence will take a lot more than just new technology at the point of generation. It will take the design of a much smarter network of distribution.

Everyday I try to look at the Google Analytics [GA] data for my business web site -- orgnet.com -- and my blogs.

GA provides various export options of the data it presents to webmasters, so I have started to export some of the interesting views as CSV files which I then import into InFlow for social network analysis.

Using the GA data we see some interesting maps. Here are the 10,500+ web pages[red nodes] that point to a page on the orgnet.com site. Using InFlow's auto-arrange algorithm we see some simple clusters of our data. In this map the popular pages are self-evident.


What is more interesting is to see which pages have multiple incoming links to orgnet.com and which pages of ours they point to. These are our true fans, who really like our content. The map below show external pages/sites [red nodes] that have 3 or more links into orgnet.com -- the green nodes are the orgnet.com pages they link to. Again, I used the auto-arrange function to display an emergent ecosystem of our pages and the pages that support them. Green nodes [my web pages] that have similar incoming patterns arrange themselves near each other.


What would be nice is to map the two-step incoming links into my site. I am also investigating other mappings with the GA data -- further musings may be posted.

Excellent new music from Brian Eno & David Byrne... free stream... enjoy!



Update: Ed V finds possible social network tracking code in embedded app... which of Valdis' friends enjoy this music enough to pass it on to their friends? See comments below.

As we approach the 2008 U.S. presidential election, we notice new books focused on Barack Obama. A few of the books are negative on Oabma and one is positive. The books do not not seem to be making much of a difference. The anti-Obama books are being read by those already predisposed against him and the pro-Obama book is solidly in the group supporting him. The din from both sides grows louder -- yes he is, no he isn't, yes he is, no he isn't ... Interestingly, NO currently popular political book, either pro or con, is written about the other candidate, McCain.

An interesting observation, from our social network analysis of political book buying data, is that none of the most influential books in the network of political bestsellers have anything to do with the upcoming election! What Happened, a look at the failings of the current presidency remains the most influential book. A close second in influence, The Post American World, a view into the rest of this 21st century, looks beyond immediate conditions to larger and longer term international trends.

I can't tell you how many times I have seen this situation: company downsizes/rightsizes, becomes more "lean and mean", and then management scratches it's collective head when things start to fall apart.

Let's look at a simple example. Here is an organization before downsizing... two people are connected with a green link if they have a reporting relationship.


The emergent work flow is much different than the hierarchy. Here is how the work really gets done... two people are connected with a purple link if they actually work together.


Pruning boxes off the organizational hierarchy is easy when you only look at budget numbers, employment costs, and formal job descriptions. The Downsizing Task Force did not know the internal dynamics [emergent work flows, tacit knowledge exchanges, etc] of Andre's organization. Here is the organization after downsizing...


Fernando and Garth have been let go. "We have eliminated the high cost employees" management proudly proclaims! Maybe so, but they have also eliminated their own year-end bonus.

Here are the established work relationships now...


Of course the Downsizing Task Force is blind to this view of the organization, so they do not see obvious solutions like we do. Without social network analysis they will be scratching their heads for a while... what went wrong? ...what do we do?

Glad they made their RIF numbers!

When choosing a map, which do you prefer — pretty or useful?

In an ideal world I would take pretty useful, but forced to choose between the two I'll take useful.

Here are two social graphs taken from my Twitter following data.

The first map, by TweetWheel, is pretty, has nice colors, a simple and elegant interface, and a nice circular layout.



But what does it really tell me? What knowledge do I gain by looking at it?

The second map, by InFlow, is not as sexy, uses less color, and produces some complex emergent patterns.



Yet, this second map gives me more information -- it shows me emergent patterns in the data [both graphs use the same following data]. Using arrowheads, the InFlow map shows me who is following whom within the community. This network layout shows the emergent communities of interest found in the data. It tells me I am not just following one theme here.

Both maps have my node and my link data excluded for ease of readability. From the second map I see that I have chosen to follow people in three emergent groups [the gray nodes are just satellites of the purple group].

By looking at who is in the group I can easily label them. The top group [ClevOH] are my colleagues in various economic development projects in Cleveland and NE Ohio. The middle group I labeled the Digerati. This is a dense group with most members following most others within the group. I see many redundant links here -- I could stop following several of these folks and probably not miss much -- since they are mostly following each other and probably aware of the same information. This group has a few satellites -- they connect to only one or two nodes in the group and therfore are not full members.

The bottom group is well connected to the Digerati, but they do have a clustering of their own. These are well known consultants in Knowledge Management, Social Networks, Organization Development and Management. Once the satellites on the right see this diagram, they may choose to follow the blue nodes on the bottom since they have much in common. Twitter networks evolve from people watching how others are connected, and then exploring the unkown person's tweets to see if they are worth following.

After viewing the last map, I have a new connecting strategy for myself on Twitter.

• need more diversity of info/topics/knowledge to monitor
  
• less Digerati, remove several redundant nodes
  
• more Consultants, more interaction with peers and elites in the consulting world
  
• few more local folks, but this cluster is pretty good
  
• look for conversations around electronic music, my hobby
  
• weave a cluster around social network analysis
  
• maybe add a little bit of randomness?
  

The first map is very easy and fun. The second map requires more work... but you get out what you put in!

What does your Twitter social graph tell you?

Update: Perhaps this is interactive Twitter social graph would be pretty useful — if it had your data in it?

I used to think Twitter was stupid.

After one week of actual activity, I find it useful.

Twitter is a "micro-blogging" platform that allows you to quickly post short messages[tweets] of < 140 characters. One of my tweets is seen in the graphic above — @dweinberger is the username of another Tweeter. Their power lies not only in their brevity but in their ability to link to other tweets and any other internet content. Twitter's concept is based on the question of everyone answering, in 140 characters or less: "What are you doing?"


I see "What are you doing?" as the wrong question — it focuses too much on daily minutiae, and not on what others may find interesting about you. We still see many tweets of people answering that question faithfully... "I am at the corner of X and Y waiting to meet my friend Z" ... "I am backing up my hard drive"... "reading the morning paper with a perfect cup of coffee". This is info you may want to know from your intimate others, but not from everyone you are reading on Twitter.

I ignore the standard Twitter question and instead use: "What are you paying attention to?" or "What do you find interesting and useful?" Judging from many of the posts I have read, other Tweeters are also using these questions as a guide to post by.

In Twitter you can not read what everyone posts, nor would you want to. You have to select who you want "to follow" — whose posts you want to read. Before choosing who I want to follow, I read a page or two of their tweets and see if they are posting interesting items. Most people leave their posts open to all potential followers, but Twitter does allow you to restrict the reading of your posts to only the people you approve. This is useful for businesses and families and other intact groups that want to limit the conversations to "within the group only". Several consultants also limit their tweets to "clients only" — I do not — see the sidebar of this blog to see my 5 most recent tweets. Some people choose to follow only their friends or acquaintances. I choose to follow who is interesting and who is posting useful information whether I personally know them or not. As we have learned in social network analysis -- it is useful to have "weak ties" to people active in social circles and knowledge pools that are different/complementary to our own.

Twitter is also a great place to ask questions — especially if you have a diverse group following your tweets. Many consultants and analysts find Twitter a great place to get quick answers — anything from how to network Windows and Macintoshes to a citation of an old paper. Journalists often troll the Twitter-verse for story ideas and people to interview.

Twitter is amazing playground for people like me who are interested in social network analysis. Twitter provides all sorts of social network metrics — focus on prestige metrics — on each person's page!

• Following: who you chose to listen to/follow = outdegree
  
• Followers: who chose to listen/follow you = indegree
  
• Favorites: the "chosen few" of who you follow = strong ties
  
• Updated: how often you post a message = network activity
  

Twitter also provides an API for those who want to get access to more data or build an application to work with Twitter. Twitter just purchased a company who had built a nice search engine of the topics being talked about on Twitter, like who is discussing social networks? Or, who is talking about the always interesting Clay Shirky or Duncan Watts? One of Barry Wellman's students was twittering in class — as many students do.

You not need a computer to access the Twitter-verse. Many people send and receive tweets from their mobile phone using their texting service or a mobile web client like Twitteriffic, which works great on my iPod Touch or an iPhone. Mobile devices make it easy to report "news" happening in front of you.

And last, but not least, there is the art of re-tweeting — broadcasting to your Followers what you found interesting/useful from one of the people you follow. By doing this you play the role of Connector by bridging your Followers to another person [Maven] they may not be following. In the networked world, you want to have the reputation as a Connector!

Join me in the Twitter-verse. It is both fun and useful.

Could you be a "person of interest"?

You might be, and not know it, and not know why.

Last week, the US Congress passed the new FISA legislation. This new law gives the government expanded powers to listen in on anyone they consider a terror suspect, inside or outside the US. While this new law aroused privacy and civil liberties questions, the law's supporters were quick to defend it, and brush off any criticism.

There is nothing to fear in the bill, said Senator Christopher S. Bond, the Missouri Republican who was a lead negotiator, “unless you have Al Qaeda on your speed dial.”

The Senator is very wrong.

You may not have AQ on speed dial, nor be following them in Twitter, nor have their hate sites in your bookmarks, nor have accepted them as a friend/contact on Facebook/LinkedIn, nor be living in the same building as them, but you may still be in their extended social network neighborhood!

See the simple social network map below. Suppose you are the green node at the center, your friends and family are the blue nodes and their friends and family are the grey nodes. This is everyone in your social network within 2 degrees of you. Most people would have dozens of blue nodes and hundreds or thousands of grey nodes in their network neighborhood.


Of all of the people in your extended network, do you know...

• them all?
  
• what they believe?
  
• what they do in their spare time?
  
• what organizations they belong to?
  
• who is in their network neighborhood?
  

You may end up being a "person of interest" if any of the blue or grey nodes in your network have any connection to a terrorist, or to someone suspected of being a terrorist. It is easy to end up in a "bad" network neighborhood — maybe your kid's soccer coach and one of your co-workers each has family back home with the same last name as a well know terrorist? And it could get worse, the watchers could easily expand their field of view and look at everyone within 3 degrees/steps of a suspect. At 3 degrees, we are probably looking at a million node network neighborhood -- easier to be in the same neighborhood as a terrorist then!

By following daily communication, and using current social network analysis methods, the watchers should eventually figure out where the covert clusters of corruption are [if any]. Yet, your life can be negatively affected by the wide net that is being cast. Who wants to be the next Richard Jewell, be falsely accused, have law suits filed against you, and have a "trial by media"?

This year marks 21 years since I started developing social network analysis software.

The first version I developed ran on a upgraded original Macintosh with an external 20MB hard drive and 512KB [not MB!] of RAM -- I still have the original machine, and dozens of floppies with various versions of the original code. When IBM came calling, "InFlow" [short for information flow] was ported from the Macintosh to Windows 3.1 and OS/2.

In 1996 [some of you were still in high school], I was fortunate to be invited to write about my early social network analysis experiences in Esther Dyson's prestigious high tech newsletter, Release 1.0. Here are my early experiences with social network analysis in large organizations.

Enjoy!

Karl Rove and I do not agree on much.

Yet, his op-ed in today's Wall Street Journal does provide an opportunity for overlap.

Rove discusses Obama's 2008 campaign strategy...

"For starters, Barack Obama's manager admitted to the New York Times that he wanted an "army of persuasion" modeled explicitly on the massive Bush neighbor-to-neighbor[emphasis mine] "Victory Committee" of '00 and '04. Those efforts deployed millions of volunteers to register, persuade and get-out-the-vote.

Sen. Obama's organizational emphasis wisely avoids the Democratic mistake of 2000, when Donna Brazille's plea for a stronger grassroots focus was ignored by the Gore high command."

Yes, all politics is local... and social. This is what I discussed in the white paper & book chapter: "It's the Conversations, Stupid!"

"...more than 45,000 canvassers – many hired from temp agencies – to register and turn out voters. It was the wrong model: Undecideds are more likely to be influenced by those in their social network than an anonymous, low-wage campaign worker [emphasis mine]."

Right on, Karl. The strategy of friends talking to friends beats the strategy of strangers talking to strangers — as I described in an earlier blog post from the 2008 presidential primary.

"The Obama campaign is trying to catch up with the GOP's 'microtargeting' program, which uses powerful analytical tools and extensive household consumer information to focus on prospects for conversion and extra turnout help."

And with warrantless wiretapping, the Bush Administration now has very good social network link data on all those "microtargets"! I jest, of course — there have been NO documented cases of counter-terrorism data or methods being used in political campaigns, but... that day is coming.

The bottom line is: the better you know both the nodes and links in the network, the better you can devise a strategy for one local voter to influence another. Help your avid supporters influence their local network.

The map below shows a social network. The grey links show: who talks to whom about politics. The nodes are colored by who they are leaning towards: red = Republicans, blue = Democrats, grey = Undecided. We are ignoring independent candidates in this simple example. How might the Undecideds tip based on the social ties illustrated?

Google has just filed a patent called NETWORK NODE AD TARGETING. Basically, the business method patent is to find the influencers in a social network and place ads on their pages/profiles/sites. In the diagram below, taken from the patent application, we see the steps in the business method.


The field of social network analysis[SNA] has much prior art in the first four boxes [405 thru 420]. Much of the SNA experience is with off-line social networks, though on-line social networks are being mapped and analyzed with increased frequency since the late 1990s. The blogosphere has been a popular source of open source data for social network analysis.

The value-added for Google is to place the electronic ad with the most influential person(s) in the network. Pharmaceutical firms have been doing social network analysis within physician networks since the mid-1960s. Big Pharma has always recruited the most influential doctors to suggest brand new drugs to other doctors in their social circle. And, companies like Visible Path have been selling social network discovery to clients for many years.

So, what makes this Google patent novel and non-obvious???

Happy Independence Day!

So, how independent do you feel?

Most people enjoy the freedom and independence of the Internet. You can go anywhere, read anything, watch anything, listen to anything, almost limitless freedom to do what you please.

On the Internet we make many choices... what to read, what to write/post, what to download, what to watch, what to subscribe to, what to bookmark, what to join, and so on.

We reveal who we are by the choices we make.

An example of these revealed patterns is the previous post on political book purchases.

Yet, as we enjoy the freedom of the net, we are being watched, tracked, mapped and analyzed. We leave a clear set of digital footprints everywhere we go and now we even have GPS and our cell phones to track us off the net. We are not as free nor as independent as we think. It is not just about privacy, but about freedom to choose and the freedom to act.

The precipice we are on, was revealed this past week when a Judge ruled that Google had to hand over, in a legal case, all of the log-in identities and IP addresses [if you have a cable modem you have a static IP address that can be easily tracked to your household] of the computers accessing material on YouTube. Yes, a record of your personal ID and everything you have watched. Wonder what patterns a marketer, psychologist, investigator, or worse -- a hacker, would find in your video choices?

"....for each instance a video is watched, the unique "login ID"
of the user who watched it, the time when the user started to watch
the video, the internet protocol address other devices connected to
the internet use to identify the user's computer ("IP address"), and
the identifier for the video."

We reveal who we are by the choices we make.

Google, is resisting the release of the information -- they would prefer to anonymize the user data before handing it over. Viacom, the other party in the suit, is working with Google and the Judge to minimize any use of your private information. The Judge has stipulated that Viacom can not use the data for marketing, nor harassing you for watching John Stewart on YouTube, instead of their preferred channel. This is a good thing.

We are walking a thin line here. It is just a matter of time before the dots that can not be currently connected, will be connected in the future -- and the key dot is your verified identity. Your autonomy will erode. Your behavior will change. Those being watched act different.

We reveal who we are by the choices we make.

Today's marketing and junk mail is based on obvious connections gathered from public information. Tomorrow's marketing and tracking will be based on private information derived from the choices you make, connected to your various on-line and off-line identities. You will be figuratively naked in front of people you do not even know. Such scrutiny will not just affect privacy, but your autonomy of choice and action.

So, on this Independence Day, be independent, yet be careful.

I have been doing a social network analysis of the purchase patterns of political books since 2003. A quick analysis of Amazon's sales data of political books gives us a highly similar analysis to that of political pollsters & pundits. In the past we saw a divided nation in our book buying data. We saw then a distinct red cluster and a distinct blue cluster with very little holding them together in terms of cross-links or books in common.

Now, in June 2008, after the major party candidates have been selected via the long primary season, we again probe the predictive patterns of political polemics. Obama says we are one nation -- not divided into blue and red. McCain proclaims his purple "maverick" roots [purple is mix of blue and red]. What does the book data tell us?

In the network maps, two books are connected if Amazon reports that they were frequently bought together or by the same person. I don't arrange, nor color the nodes before feeding the also-bought data through the InFlow software. The software has an algorithm that arranges the layout of the nodes based on each node's connections, both direct and indirect. Once the software finds the emergent pattern, and any clusters, I review the books in those groups and then see whether they are blue, red or purple.


Once the map was completed, I ran InFlow's network metrics to see which book(s) were most influential in June 2008. Not surprisingly, McClellan's What Happened came out on top, followed closely by Zakaria's The Post-American World.

The purple books [neither Right nor Left] were hard to distinguish this time. According to the network layout algorithm, they are closely integrated with the blues [Left]. Some of the books that ended up purple were surprising. George Will and Patrick Buchanan are outspoken conservatives, I expected them to show up in the red cluster. Maybe this reflects the split we have seen on the Right between the "old conservatives" and the "neo-cons"? The buying data shows that the old conservatives have more overlap with the progressives than they do with the neo-cons! Even Ron Paul's and Jesse Ventura's books link more with the blue than with the red.

Is the country moving from slightly right of center to slightly left of center?

Update: After viewing the network, Micah Sifry @ Personal Democracy Forum, revealed that he sees a simple pattern -- the map shows who is for the Iraq War and who is against. The red authors & readers are for the war, while the blue & purple authors & readers are against the Iraq War.