Monday, January 19, 2009

The story behind the SMART Table

Heather Ellwood wrote a feature article about the invention and innovation process of the SMART Table in this month's EdCompass feature.

It's an interesting story that includes interviews with both the SMART Table development team and the reaction from a Calgary Board of Education Elementary classroom.

We've been thrilled with how quickly the SMART Table has gone from research prototype to shipping product.

Friday, December 14, 2007

Innovation at SMART Technologies

When I started my job search a few months ago, I was looking for a position that would allow me to learn about the process of innovation in a real world setting. In particular, I wanted to learn how to take novel ideas and bring them to product.

My PhD has focused mostly on invention: coming up with novel ideas and pushing the boundaries of what can be done with computers. While I had taken courses on innovation, starting a business, and marketing ideas I wanted a venue where I could apply these skills in practice.

After a long period of deliberation and negotiation with people from companies all around the world (including U.S.A., Holland, Japan, and Finland) I have decided to accept a position at Smart Technologies. I strongly believe that Smart Technologies will provide an excellent opportunity for me to learn how to transition novel ideas from conception to actual product. Check out this post by Executive Chairman David Martin on innovation.

In addition, I have decided to accept the Alberta Ingenuity Research and Development Associates award. As reflected in my previous post, we already have significant external validation about our future collaborations.

Dr. Gerald Morrison (External Research Manager at Smart) was instrumental in collaborating with me on the Alberta Ingenuity award and the job offer from Smart. I would like to express my deepest thanks to him for his support.

On that note, Gerald was interviewed by the Canadian Broadcasting Corporation (CBC) today. When asked about the benefits of Smart's collaboration with our lab, Gerald mentioned the recent hire of Edward Tse. If they use this clip on national television, this will be a very public announcement of my new position!

Wednesday, December 12, 2007

AI Research Associate Approved!

Good news! I just got my Alberta Ingenuity Industrial Research Associate application approved with glowing results. Alberta Ingenuity uses the ProGrid evaluation system to rank candidates who apply for a particular scholarship. I learned that my scholarship was one of the top ranked in terms of benefits and initiative. Chart 1 is a 2D plot of my ranking (square) to all other applications of this scholarship. It is the highest in terms of benefit and tied for second in terms of initiative.
When I looked at the proposal profile I found that all four categories relating to the candidate (letters of reference, academic record, career track record, and candidate suitability) were given the maximum possible relative strength.
A quote from an Evaluator's comment on my application:
Edward has glowing letters of refrence commenting on the knowledge, skills, and ability relevant to this opportunity. He is an outstanding, award-winning researcher with numerous competitive awards and independent media response to presentations and achievements. He is quickly emerging as a star player in this field and is a strong match for this opportunity.
Now I need to decide if I will accept this grant or not...

Wednesday, November 07, 2007

Herr Doktor Tse

I'm happy to annouce that I've successfully defended my doctoral dissertation! The defense presentation and question rounds went surprisingly smoothly. After my defense friends from the Interactions Lab presented me with a little surprise.

I received a custom made hat with a scene depicting my PhD research on speech and gesture interaction over digital tabletop displays. Apparently this is a tradition done in many German Universities and started in our by some of our graduate students from Germany.

The theme of my hat is focused around speech and gesture interaction around a table. It has a tabletop display with two chairs (made of foam core) and a wooden mannequin attached connected to a series of strings. Pulling the strings on the tassle causes the mannequin's arms to move and also plays a sound.

Sound was achieved by using a Napolean Dyanmite gift card. The hat says a quote from the movie whenever the tassle is pulled "So like, nunchuck skills, bow hunting skills, computer hacking skills". Some matching text was added around the base of the hat that says "critics agree, you got flippin sweet PhD skillz!"

This was certainly a very pleasant surprise, thanks iLab!

Thursday, August 30, 2007

Digital Tables as a Disruptive Technology

I've been reading The Innovator's Dilemma by Clayton M. Christensen of Harvard Business School and I've been thinking about how the lessons of his book would apply to disruptive technologies such as interactive wall and table displays.

Essentially, this book argues that companies (such as the Hard Drive industry) are exceedingly good at listening to their customer base and sustaining innovations in their products to meet their client's needs. Disruptive technologies (e.g. smaller hard drives, solid state disks) are usually first developed and marketed by large corporations. However, these technologies usually do not meet the needs of existing clients (e.g. too small size, too expensive, too slow) thus these projects are shelved until the technology becomes more viable in the marketplace.

Microsoft Surface
On the flipside, startup companies focus on niche markets (e.g. laptops, mp3 players) and ignore the needs prized by current vendors. Often the rate of improvement of these technologies greatly exceeds the demand by their niche market. This happens until the technology improves to a point that it can compete with the sustaining technologies of the large corporate vendors (e.g. 3.5" disk drives).

Hewlett Packard
At this time we see a huge shift in the industry, people start migrating towards the disruptive technologies and the large corporations are continually catching up to the latest advancements of the startup companies. This continues to happen in many areas such as the Internet, PCs, GUIs, industrial hauling, cell phones, portable audio, PDAs, laptops and much more.

Panasonic
Digital tables are a disruptive technology, right now they are not as efficient as using a keyboard and a mouse. Speech and gesture recognition have years of improvement ahead of them before they become viable for everyday use. However, the rate of improvement of interactive surface technology is significantly faster than the rate of improvement of desktop interaction technologies.

Hitachi
Eventually the performance of interactive (small and large) surfaces will rise to challenge the efficiency of desktop computers. This will not happen overnight, it could be decades before we see such a transition. Nonetheless it will happen. And when it does I wonder if the existing companies will rise to deliver these next generation technologies, or if it will be startup companies that will once again lead the way in technology innovation.

Smart Technologies
As I enter my second and third round of job applications, I have applied to many large corporations that are adamant about making sure they are on top of the large display interaction trends. Large corporations such as Microsoft, Hewlett Packard, Panasonic, Hitachi, Smart Technologies, Philips, Mitsubishi, Accenture, and Sony have already created demonstration systems of interactive large displays.

Philips
Several startup companies have also been created to market large display interactions to niche audiences. These companies include Perceptive Pixel, Lemur Jazz Mutant, and Fingerworks which was recently acquired by Apple for the development of the iPhone which was later replicated by Nokia).

Mitsubishi
The billion dollar question is: will it be the startup companies or the large corporations that deliver these next generation interactive touchscreen technologies. I'm really not sure, but I intend to find out by apply to each of these companies.



Accenture









Sony









Perceptive Pixel








Jazz Mutant









Fingerworks

Tuesday, August 21, 2007

The Job Hunt Begins

Great news! I'm almost finished my PhD and I am now getting ready to apply for jobs for the fall. I'm planning on having three waves of job applications, and I just completed my first wave yesterday.

I spent the most of the day writing these emails, editing them, getting friends and family to read them and editing them some more. Basically, I had six Gmail tabs open and was constantly moving back and forth. My first wave is composed of people that I know personally and that I have met numerous times at various industry, academic and general conferences and conventions.

I sent out all six emails at exactly 10:48pm EST. That way everyone on my first wave has an equal opportunity. I jokingly say to friends that I'm an "equal opportunity interviewee."

Most of my experiences thus far have been academically focused around inventing new interaction techniques and publishing papers. During the next steps of my job search I would very much like to learn about the innovation side of this equation. In particular, what does it take to make novel technologies accessible to a general audience?

Thus, when searching for companies I am looking for companies that will advocate for the research that I am doing internally within the organization. I am also looking for my supervisor to be transparent about the needs and direction of the company so that I can better target my research. I am also interested in staying connected with the external communication (academic and industry) so that I can see how my work compares to other leading firms in the field.

My goal is not to work as a member of a team in a industry setting, where I'm interested in learning about a company's values so that I better target my own research and development.

Wednesday, July 18, 2007

Why Hollywood should hire Interactions experts

Having watched a number of movies lately, I've found the digital display interactions more and more irritating to watch. I am becoming increasingly concerned that the interactions portrayed in movies shape our expectations for how we expect to interact with computers in the future.

Seriously, I've spoken with many researchers who have complained that their work is constantly being compared to movies such as Minority Report, or The Island. This is a good thing as it means that people are becoming more aware of novel interactions for large displays, but it can also makes it difficult to convince others that the interactions shown in movies are not necessarily the best there is. For example, I've found that Minority Report has made it difficult to convince others about the benefits of direct touch interaction, where you touch directly on a large display instead of working at a distance.

Photo from Yahoo! Movies
Another recent example that I found a bit strange was Fantastic Four: Rise of the Silver Surfer. In this movie Dr. Reed Richards (a.k.a. Mr. Fantastic) uses a portable handheld computer (looks like an Archos 604) to control a large satellite located on the roof of his home. Dr. Reed comments on how wonderful the device is as he types with his two stretchy thumbs on the display. Apparently he is able to type incredibly quickly by stretching his thumbs to all parts of the display (e.g. his left thumb moves to the right side of the screen).

This doesn't seem effective for a number of reasons, I'll briefly describe two: first, the QWERTY layout of most onscreen keyboards will not be faster with longer thumbs. In fact, the opposite will likely be the case since speed is achieved by switching between using your left and right thumbs. This fast switching is not necessary if your thumb can reach the entire screen. Second, anyone with big thumbs can attest that this typically results in less accurate typing. Your thumb covers more area and thus it is hard to accurately point to a particular key. Onscreen keyboards have other issues such occlusion and lack of haptic feedback.

Now, this might seem a bit harsh for such a brief scene but I emphasize that many other movies share the same problem. Which brings me to my point, Hollywood movies would benefit from having an interactions expert on staff to help plan out and design suitable interactions for a movie.

Collaborations between interactions researchers and Hollywood have been successful in movies such as Minority Report when Spielberg and his team visited the the MIT Media Lab to learn about current projects. They took some of these interactions and showed what they would look like in a futuristic setting. The result was an interaction that wowed audiences around the world and set a precedent for future interactions expectations. People were talking about the interactions in this movie long after it had left theaters, it had a lasting impact on people.

I hope that Hollywood will continue to collaborate with interactions researchers in the future, there's a lot of interesting stuff happening in our field that we'd love to see on the big screen.

Thursday, May 17, 2007

Call me Carlo

Lol... there's someone on the Internet claiming to be me! I was reading my email when my Gmail RSS feed mentioned that there was an article on the Top Ten Issues of OLPC. I decided to read it given that I had written a similar article a few days earlier. The first paragraph said:
"I am Carlo Emmanoel Oliveira Ph.D. I am not officially involved with OLPC but their project had a great impact in my life. I am a University teacher in Brazil, partly working on educational projects to improve education in general, including children in all ages. I think OLPC is one of the more important steps to bring computers as an aid to education."
Then the rest of the article is taken word for word from my OLPC Article! Someone was taking credit for the article I had just written. The funny thing is that this link that is supposedly to Carlo's home page actually links back to my own personal blog (that has my name written all over it).

Anyways, I Googled him to get more information and found that he was in fact a University Instructor in Rio De Janiero, Brazil complete with 27 published articles, awards, and patents in the area of software engineering. Something didn't seem right here. So I emailed the people managing the OLPC News blog and got a fairly prompt response:
"Edward,

I seem to owe you & Carlo an apology.

I was in an email conversation with Carlo about a post he wanted to publish that had a couple of attachments and many links. I became confused as to the sites he was interested in responding to (yours being one of them) and items he wrote (a response to your top 10). In that confusion, I attributed your content to him and posted it on my site thinking he gave me permission to repost it. He disagreed with a few of your positions in his content, so he may be as annoyed with me as you were with him.

And I am very annoyed with myself. I dislike when others take content wholesale, with our without attribution who do not ask permission first. I apologize to you personally now. I will correct the post tonight and append an apology/correction, if I can be so bold as to ask permission to keep the content up.

Thanks for catching this"
Sounds like there was a simple misunderstanding regarding who created the article, which is odd because it is written in first person. I'm happy to let it slide and I also really appreciate the comments that people have written about my article. However, if this continues I'm going to have to start calling myself Carlo Oliveira, PhD :)

Wednesday, May 09, 2007

The Top Ten Issues of OLPC

During the past Human Factors in Computer Systems conference in San Jose, California there was a lot of attention on the One Laptop Per Child (OLPC) project (see a video). The project goal is: "To provide children around the world with new opportunities to explore, experiment and express themselves." In the mission statement the website claims that OLPC has been "extensively field-tested and validated among some of the poorest and most remote populations on earth". While this could be used in conjunction with current teaching, part of the goal is to support self-exploration without the aid of formal teaching.

We had the privilege of hearing from some leading usability researchers in developing countries about their opinions of the OLPC project. I've included those references that I could find along with some anecdotal notes that I recorded during plenary talks and individual conversations with leading researchers in the field. While other articles focus on financial/deployment issues, this article focuses on Education and how children will interact with OLPC.

This article is a compilation of ten key issues facing the OLPC project mentioned by other researchers and through conversation. There is a concern about how OLPC might fit into the larger infrastructure of education in developing nations. I personally feel that technology has a large role to play in the future of education (this is already seen with the exploding growth of companies like Smart Technologies that focus on the education market) but there is a need to understand how the technology fits within the ecology of education in developing nations.

This article is not meant to condemn the OLPC project as its aims are focused on goal that would benefit society as a whole (these comments could apply to projects such as Intel's Classmate PC as well). Rather it asks: how can OLPC be improved? Is this the right approach? What other approaches could be used? Before massively deploying such a technology, it is crucial that we have this debate.

I hope you find this article informative, please feel free to leave any comments.

10. Focus: The focus of OLPC has been completely on the technology with the goal that a new technology will change how we educate children. This is like evaluating the quality of our education based on the type of glue that is used to bind textbooks or the images on the cover pages. There is a lack of focus on education and improved learning. People dismiss the importance of teachers suggesting that comptuers and self directed learning will be a suitable replacement. Teachers, be they your peers, parents, or trained individuals are a crucial part of feedback system of learning.

9. Readability: "Many who test displays contend that in order for a display to be readable in sunlight, it must have a maximum brightness of at least 500 nits and a contrast ratio of at least 2 to 1. Some manufacturers of outdoor displays go for 1000 or even 1500 nits, but laptop and notebook screen brightness comes no where near 500 nits." [Gerber, 2005]

8. Existing infrastructure: A recent study found 97 percent of people in Tanzania said they could access a mobile phone, while only 28 percent could access a landline [Prahalad, 2004]. While OLPC does not leverage such infrastructure, a simple voting system could dramatically improve a teachers' understanding of how well their students were learning class material. Also, Internet is accessed mainly through cell phones and Internet cafés in developing nations. Thus equipping a classroom, particularly one that is not in a building (e.g., children sitting under a tree) poses serious infrastructure issues.

7. Not all learning can be done with an OLPC: Studies have shown that certain learning tasks such as mathematics are very difficult to learn using a computer keyboard and mouse and consequently result in decreased academic performance for students [Oviatt, 2006]. In particular, it has been shown that using a keyboard and mouse for solving mathematical questions requires significantly more time and results in more errors than using pen and paper. Researcher have also noticed that this decrease in performance is increased among the students that are struggling the most in the classroom as they are stuck trying to master both the course concepts and the technology at the same time.

6. Lack of content: content provision is a serious issue for these devices. If it is the expectation that teachers will produce all of their own content, using an OLPC could be more work that just buying a book and sharing it among students. Content needs to be provided free of charge. OLPC claims to be providing infrastructure but without content providers it will be impossible to use. This is the critical mass problem: what good is a fax machine if only one person in the world has one.

5. Keyboards: We need to ask ourselves what current practice is in the learning environment and design solutions that would fit the current practices of students and teachers. For example, if students are more used to using a slate, perhaps the keyboard and mouse metaphor of existing systems is inappropriate. Similarly, if people are familiar with cell phone technology it may be useful to develop systems to support their current practices with cell phones. Perhaps what we need are more (touch sensitive) slates and (digital) black boards rather than OLPCs alone [Buxton, 2005].

4. Scalability: Lets say a teacher wants to get all 49 of their students in a single class to perform a particular exercise. Given that the instructor cannot see all 49 screens at once, how do they gage if students are confused or not understanding the task at hand? Each student is looking at their own private display rather than looking at the teacher/blackboard making it harder to guage student engagement at a glance. Would it not be better to have a single large digital display than a classroom full of individual PCs? Take for example, the Smart Technologies Senteo system where each student can have a clicker to respond to polls in the classroom. The total cost of ownership would probably be less than the cost of a $100 laptop per student.

3. Ergonomics: the fact that OLPC is designed as a laptop leads to ergonomical problems as students may not have a table that they can put the computer on. Thus they will likely have to place it on their laps for extended periods of time leading to discomfort that can also hinder learning.

2. Wrong Problem: While the One Laptop Per Child project focuses on providing technology to children in developing nations the major issue affecting student outcomes seems to be the training of teachers [Vegas, 2007]. With student to teacher ratios reaching 43:1 in primary Sub-Saharan African schools with only 69% of primary schoool teachers recieving any sort of formal training it seems that technology would only exacerbate existing issues in the education system.

1. The Community of Learning vs. The Cult of the North American Individual: The name OLPC is a problem as the focus is on Personal Computers for Individuals ignoring the fact that community feedback is crucial part of learning. Self directed learning cannot be effective without feedback from peers, parents and teachers. Even when parents and peers are not available children will often huddle around a single computer to collaborate and provide constructive feedback [Pawar, et al, 2006]. Developers can push this learning configuration further by providing interactivity for each child on the same display (through multiple mice and keyboards). Studies have shown that this configuration results in students being more engaged, faster and more accurately able to do problem solving tasks [Scott, et al., 2003]. Students need a learning community to provide the feedback needed to fully understand the material they are investigating. OLPC will likely do the opposite by pushing students away from each other to their own computers.

Interview with Jeremy Wagstaff from Wall Street Journal Online

jeremy.wagstaff: Do you mind offering a general comment about the future of interfaces with regard to something like Google Earth? Your project sounds interesting, and I imagine it's the starting point for a whole new way of operating these kinds of environments

Edward: Sure, it may seem like this is a whole new way of interacting with maps but this is in fact not the case. Many of the interactions that I have tried to design over geospatial applications (e.g., Google Earth, Warcraft III) are designed to mirror the actions that people do in the physical environment. For example, studies of military command and control situations show that Brigadier Generals work together on a paper map on a table. They use multiple fingers to mark points of interest and use two hand sides to mark areas of interest.

jeremy.wagstaff: interesting...

Edward: We now have technology that can detect these unique hand postures/configurations on a digital table where people can get the benefit of panning/zooming along with real time updates. I'd envision that this kind of interaction will in fact be more similar to the way that people interact in the physical world rather than some totally new type of interaction.

One thing that is particularly interesting is support for multiple people. Current operating systems limit interaction to a single keyboard and mouse at any given time.
There are new touch technologies that can detect the touches from multiple people simultaneously. While we are currently working over existing applications (e.g., GE), researchers are working at building true multi user applications that will be able to understand multiple people are respond appropriately. For example, an annotation mode where multiple people can draw on the surface simultaneously with their own separate markers/colours

jeremy.wagstaff: that does sound cool. when might we see any of these on sale?

Edward: well, actually these surfaces are already being sold. For example, the MERL Diamond Touch is being sold through Mitsubishi Electrc Research Laboratories in Cambridge, MA. Also, Smart Technologies sells a large digital whiteboard that can support up to two touch simultaneous touches. Currently they are a bit expensive $7'000-$12'000 USD but as with all things their prices will be dropping in the near future. In fact, the price of a Smart Board has dropped significantly over the past few years and the market for large digital whiteboards has really exploded in the education sector. I'd envision that these kinds of surfaces should be affordable to the regular home consumer within 7 years. The technology is already in place, it's just a matter of having the right applications and interactions

jeremy.wagstaff: that should be cool...

jeremy.wagstaff: thanks a lot for this. can i ask a bit about your background? are you a canada native?

Edward: Yes, I was born and raised in Canada. Currently I am finishing up a doctorate at the University of Calgary. I have done internships with both Smart Technologies in Calgary, AB and Mitsubishi Electric Research Laboratories in Cambridge, MA.