ContentNotEditable: What the “death” of the mouse and keyboard means for content creators

First let me start with some disclaimers to try to make sure this post is not misinterpreted: I am not arguing that the mouse and keyboard are really dead or that the lack of a mouse on tablets is a bad thing.  I am not arguing that the ipad or similar devices are awful for education or content creation.  I am just thinking about how to make them even better in these areas, and conceptual and technological roadblocks in the way.  Some of the criticisms of the ipad as a content consumption device (here, here), have been addressed with the ipad 2 and other new android tablets with their inclusion of cameras and input ports, and some tablets are even coming out with a stylus, like the HTC Flyer.

But imagine any creative person – creating a 3d character for a game, drawing a picture, composing a music score, creating a graphically rich document or presentation, etc.  They likely have something in their hand, or their hands are busy doing something.  When that involves interacting with a computer they are likely clicking the mouse to drag something around or edit text, for example, or using a stylus on a digitizer surface (like a wacom) for drawing, or typing away on a keyboard.

These input devices are all essentially gone on new tablets and smartphones.  You can still type (slower) on virtual keyboards, and you can click like a mouse with your finger (tap).

An example of the impact of this is rich text editing, like with a word or openoffice or google docs document.  Many browser-based wysiwyg editing tools, which are used virtually everywhere, such as in moodle (which uses the TinyMCE editor) or drupal, no longer work when you access them from an ipad or iphone or android device (or other mobile platforms like blackberry or palm webos).  Even the newest “HTML5″ editors, such as Aloha Editor, pop up an error message if you try to access them from a mobile device.  Other browser-based editing and drawing tools also no longer work on these new platforms, or you have to draw with your fingers.  Most of Google’s and others’ tools like Google Presentation do not work on mobile platforms.  Really, just imagine most any software people use on a desktop to create stuff – like office, or the flash ide, or gimp/photoshop, blender 3d, etc.  Even when programming, which really is just typing in plain text, we usually prefer to use IDEs that popup suggestions and corrections to help us out.  For many of our desktop apps its hard to even imagine them working on a tablet or phone.

The rich text editing tools in browsers like TinyMCE or CKEditor primarily rely on the contenteditable HTML attribute to support editing.  Add that attribute to an HTML element, and the contents of that element become editable inside the web browser.  It works in all browsers, including old Internet Explorer versions. It doesn’t really work well or even at all on mobile browsers though (see hereherehere, here).

Newer versions of android, webkit, and mobile firefox have been slowly improving their support for contenteditable, and maybe they will eventually “fix” the issue, but I’m not sure that this will be fixed through engineering alone.  Some code editors like codemirror 2 and the ace editor are trying out workarounds like using a hidden textfield that captures key presses.  Codemirror 2 works on an ipad somewhat, the ace editor does not.  It remains to be seen if a similar trick might work for a rich text editor (it’s tricky enough just to do it for plain text).  And like I said, the HTC Flyer and other tablets (esp. those being designed for medical and other professionals) are starting to include a stylus, and it remains to be seen if that will catch on (it didn’t before with older tablets).  Others are coming out with dual screen tablets, where the second touchscreen can work like a touchpad on a laptop or nintendo ds, but that also may not catch on.

Another more general alternative strategy to this issue of tablets having no mouse or other input devices other than the touchscreen and the camera (which can be used for gestural or other input), might be to conceptually rethink how to support multimedia creation on these mobile platforms.  Perhaps we should drop the notion of “documents” or “pages”.  After all, you don’t think of a flash widget as a page or document.  You don’t think of a game as a set of pages or documents.  And Apple and other developers have already created apps for some specialized types of content creation and creativity, such as musical instrument simulators and so forth.

So, this may be a pre-paradigmatic moment where we’ll see what catches on: will we try to perfectly “emulate” the mouse and stylus and its supported interactions via other means such as gestures, or will new and unique types of interactions continue to catch on (like multitouch stuff).  Probably a combination of both, but so far the camera is hardly being used at all for input, other than recording videos or taking pictures.  And some may dismiss the idea of a stylus ever catching on again, but Apple has occasionally made “mistakes” before (the first mac didn’t have a floppy drive, for example), and others have been successful in incorporating a stylus, like the Nintendo DS, which my little boy continually loses :)

About these ads
Posted in android, computers, development, edtech, html5, opensource, software
2 comments on “ContentNotEditable: What the “death” of the mouse and keyboard means for content creators
  1. [...] his most recent post, which really got me thinking – “What the “death” of the mouse and keyboard means for content creators”: First let me start with some disclaimers to try to make sure this post is not misinterpreted: I am [...]

  2. edtechdev says:

    Wired had a related post just come out: The iPad Falls Short as a Creation Tool Without Coding Apps

    http://www.wired.com/gadgetlab/2011/03/ipad-creation

    Here is my comment (on hackernews):

    This problem probably won’t be solved by Apple, but it will be solved by others who are working on browser-based IDEs right now.

    There are several browser-based programming sites such as http://www.playmycode.com/, cloud9, http://jsdo.it, jsfiddle, etc. Even Eclipse is coming out with a browser-based version soon (project orion). Now, some of these don’t work on mobile or tablet devices yet (broken contentnoteditable in the browser), but they will. See for example the codemirror 2 editor, which does work on an ipad, albeit not perfectly: http://codemirror.net/2/

    A related issue is what the author talks about – it’s harder and slower to type on an ipad or smart phone than a device with a regular keyboard. I don’t know of a solution to that yet. Perhaps it will involve better touch-driven keyboards (sort of like swype and related options – but tailored for programming), or perhaps using voice or the camera (gesture recognition) for input, or perhaps we’ll just adapt and get used to the current on screen keyboards – kids aren’t really complaining about them, and are pretty fast at using them, but there’s a difference between typing a text message and programming with all the special symbols and indenting used (a pain without a regular keyboard). Or perhaps more scratch-like programming environments will gain popularity (program by dragging and dropping blocks instead of typing).

    My own solution is to create a browser-based development environment and programming language that compiles to javascript, but the language is designed to be easier to type in using a standard mobile keyboard or even voice input. Still vaporware, sorry, but it’s sort of like a cross between ruby and visual basic (cue snarky responses).

Comments are closed.

Doug Holton

Doug Holton

developing educational technology

Archives

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 3,383 other followers

Follow

Get every new post delivered to your Inbox.

Join 3,383 other followers

%d bloggers like this: