Category: Seminar/Lecture Notes

Image taken by the Times Union.

Yesterday I attended the presentation of “More Like Us: Human-Centric Computing” by Craig Mundie, the Chief Research and Strategy Officer for Microsoft. I hadn’t actually thought that I would be able to see it, since it ran through not just one, but two of my classes. However, due to interest in the event by the class, the professor of my first class decided to end class early so we could head to EMPAC to listen to the presentation. I did still have to leave early for my second class, but he had already finished the main presentation and moved to Q&A by then.

After being introduced by President Jackson, he started the presentation with a short history of how there have been several big shifts in computing for human use, starting with the use of microprocessors to enable many more people to be able to have computers, and ending at the current era of GUIs (graphical user interfaces). This was his segway into the main purpose of the presentation, which was to show some of the ideas currently being worked on to try to expand out from GUIs to what he called NUIs (natural user interfaces). He presented several interesting vignettes, all of which used variants of the soon to be released Kinect system. Someof the more interesting ideas were an already tested use of a gesture system to allow doctors to scan through images/media without having to make contact, which helps to maintain a sterile environment, and an automated shuttle scheduling AI, which I’m sure some people on campus would love to have (or rather, off campus). This used an older prototype of the Kinect system, which much of the demo revolved around. It is equipped with various sensors allowing it to process and use data about the entire body/movements of 1-4 people, as well as voice commands, all of which he demonstrated in some apps planned for Kinect.

Although the system will be sold as a gaming platform, the examples given before the demo of real-world applications was a good reminder of just how versatile such technology would be. He also spent some time talking about some of the challenges they faced while making it, such as getting all of this processing to run using only a few percent of the system’s available memory, to leave as much as possible for the game developers’ use, and keeping the construction cheap enough to sell the system at an affordable price. He also discussed some issues that the sensors had to deal with dynamically, such as the (I forget if this was the actual name..but it’s close) “Annoying little brother” problem, and similar situations where additional limbs, movement, and objects enter the sensor’s range, as well as how the sensors try to infer where body parts are, even with obscuration. The only term I recall on this inferencing was that it used advanced Bayesian reasoning, a term that I’ve heard before but don’t know any real details about.

As far as semantic technology goes, this presentation did not really touch on that area, but he did mention semantics while he did a demo where he was navigating a virtual ‘wish list’, which was a good example of where such work would be helpful. He had mentioned that the ‘wish list’ system used semantics to help sort the items on the screen into various categories, such as pasta machines or bags, and I think it could probably be extended to things like searching by color, use, location, or other things. Clearly, this is an example very similar to ones used when explaining ontologies. Also, this part of the demo involved several voice commands, which he used fairly colloquial language with successfully. I don’t know how versatile its language processing actually is, but from the demo it seemed robust, and probably also involves a lot of language semantics.

It was an interesting presentation about future ideas related to increasing the ease of computer use and computer integration. The only frustrating part was not on his part, but the first question that was asked by a student, which was also unfortunately the only question I had the bad luck to witness. He basically just stood up and said that he thought the technology was clumsy and that it looked just like the Wii. He even went on to claim that everyone in the room agreed with him. He ignored the fact that the entire presentation was about future ideas and uses/refinements of this proto-technology and the fact that a Wiimote, where a custom remote is tracked by specific signals, is totally different from a system that is processing not only your voice but also the movements of your entire body without any colored patches, electrodes, or signalers on your person. I hear similarly unenlightening questions plagued the Q&A, unfortunately.


Every Wednesday there is a Cognitive Science Seminar, where a speaker comes in to give a lecture/talk. This is where I first heard about the TWC, and today was again about some of the goals, considerations, and early applications towards the development of the semantic web, presented by Jim Hendler, one of the TWC professors.

The first point was about what job the computer is expected to play in the developing web. Put simply, the computer should be able to do administrative activities by itself, such as reading/organizing/managing data, which will free up people to do the creative activities. He goes back to this point several times later, when looking at current online communities. One example shown with this was Google Trends, specifically the flu trends, where the computer was the one processing and producing these trend graphs by itself, allowing people to respond faster and use this data.

Another big recurring point was the distributive nature of these applications, where the data is not just limited to one site or one source, but produced and usable as a collective project. The main example used throughout the presentation was GalaxyZoo, where people can spend a few minutes identifying what types of galaxies are being observed in deep field images. This has resulted in a much faster identification process, helps developing astronomers, and more. Another example was reCaptcha, which actually helps work in computer reading and processing of old documents, by using a distributed process of people entering in captchas that are actually drawn from various fragments that the program is analyzing.

Current social applications/communities like Wikipedia and Facebook were used to look at some of the issues with dynamics, trust, and governance.  He talked mostly about trust and governance using the example of Wikipedia.  It’s trust issues are well-known, where issues rise up on a web where anyone can post anything and change information anytime.  The checks in place against this lead to issues of governance.  In terms of governance, it is more of a hybrid open-closed system rather than the completely open system it is usually thought of having, where the framework is open-source, and the data is freely editable and available, but the governing systems on organizing and managing and protecting that data are proprietary.  Things like trust/governance are thus not shared with other users of MediaWiki. 

He also listed engineering problems with these approaches. One problem is the creation of tools to help create/share/evolve, and here he seemed to be referring to the earlier point about governance models, and how they are not typically shared, causing new sites to have to make their own each time, although the framework might be available.  This is similar to another problem listed, with making sure protocols fit with social expectations, referring to the transparency, awareness, and accountability of the rules/guidelines. Finally, the architecture itself must be looked at, referring to the implementing of its distributive, open, and dynamic properties.