Damian Mehers' Blog Xamarin from Geneva, Switzerland.

30Aug/151

Android 5.0 Media Browser APIs

When I read the release notes for the Android 5.0 APIs I was delighted to see this:

Android 5.0 introduces the ability for apps to browse the media content library of another app, through the new android.media.browse API.

I set out to try to browse the media in a variety of apps I had installed on my phone.

First I listed the apps that supported the MediaBrowserService:

  private void discoverBrowseableMediaApps(Context context) {
    PackageManager packageManager = context.getPackageManager();
    Intent intent = new Intent(MediaBrowserService.SERVICE_INTERFACE);
    List<ResolveInfo> services = packageManager.queryIntentServices(intent, 0);
    for(ResolveInfo resolveInfo : services) {
      if(resolveInfo.serviceInfo != null && resolveInfo.serviceInfo.applicationInfo != null) {
        ApplicationInfo applicationInfo = resolveInfo.serviceInfo.applicationInfo;
        String label = (String) packageManager.getApplicationLabel(applicationInfo);
        Drawable icon = packageManager.getApplicationIcon(applicationInfo);
        String packageName = resolveInfo.serviceInfo.packageName;
        String className = resolveInfo.serviceInfo.name;
        publishProgress(new AudioApp(label, packageName, className, icon));
      }
    }
  }

The publishProgress method updated the UI and soon I had a list of apps that supported the MediaBrowserService:

Apps that support MediaBrowserService

Next, I wanted to browse the media they exposed using the MediaBrowser classes:

…
public class BrowseAppMediaActivity extends ListActivity {
  private static final String TAG = “BrowseAppMediaActivity”;
  …
  private final MediaBrowserConnectionListener mMediaBrowserListener =
      new MediaBrowserConnectionListener();
  private MediaBrowser mMediaBrowser;

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    …
    Log.d(TAG, “Connecting to “ + packageName + “ / “ + className);
    ComponentName componentName = new ComponentName(packageName, className);

    Log.d(TAG, “Creating media browser …”);
    mMediaBrowser = new MediaBrowser(this, componentName, mMediaBrowserListener, null);

    Log.d(TAG, “Connecting …”);
    mMediaBrowser.connect();
  }

  private final class MediaBrowserConnectionListener extends MediaBrowser.ConnectionCallback {
    @Override
    public void onConnected() {
      Log.d(TAG, “onConnected”);
      super.onConnected();
      String root = mMediaBrowser.getRoot();
      Log.d(TAG, “Have root: “ + root);
    }

    @Override
    public void onConnectionSuspended() {
      Log.d(TAG, “onConnectionSuspended”);
      super.onConnectionSuspended();
    }

    @Override
    public void onConnectionFailed() {
      Log.d(TAG, “onConnectionFailed”);
      super.onConnectionFailed();
    }
  }
}

I’ve cut some code, but assume that the packageName and className are as they were when queried above. No matter what I did, and which app I queried, the onConnectionFailed method was invoked.

Here is the log from when I tried to query the Google Music App:

29195-29195/testapp D/BrowseAppMediaActivity﹕ Connecting to com.google.android.music / com.google.android.music.browse.MediaBrowserService
29195-29195/testapp D/BrowseAppMediaActivity﹕ Creating media browser …
29195-29195/testapp D/BrowseAppMediaActivity﹕ Connecting …
16030-16030/? I/MusicPlaybackService﹕ onStartCommand null / null
16030-16030/? D/MediaBrowserService﹕ Bound to music playback service
16030-16030/? D/MediaBrowserService﹕ onGetRoot fortestapp
16030-16030/? E/MediaBrowserService﹕ package testapp is not signed by Google
16030-16030/? I/MediaBrowserService﹕ No root for client testapp from service android.service.media.MediaBrowserService$ServiceBinder$1
724-819/? I/ActivityManager﹕ Displayed testapp/.BrowseAppMediaActivity: +185ms
29195-29195/testapp E/MediaBrowser﹕ onConnectFailed for ComponentInfo{com.google.android.music/com.google.android.music.browse.MediaBrowserService}
29195-29195/testapp D/BrowseAppMediaActivity﹕ onConnectionFailed

Notice the message about my app not being signed by Google on line 7?

I’m assuming that only authorized apps are allowed to browse Google’s music app such as Google apps supporting Android Wear and Android Auto, but not arbitrary third party apps. Indeed the documentation for people implementing MediaBrowserService.onGetRoot indicates that:

The implementation should verify that the client package has permission to access browse media information before returning the root id; it should return null if the client is not allowed to access this information.

This makes sense, but it is disappointing. Just as users can grant specific apps access to notifications, it would be nice of they could also grant specific apps the right to browser other apps' media.

Please let me know if you discover I am wrong!

Filed under: Android 1 Comment
23Aug/157

Using Android Wear to control Google Cardboard Unity VR

Using a VR headset, even one as simple as Google Cardboard, can be mind-blowing.  Nevertheless it is the little things that can also be disconcerting.  For example looking down and seeing you have no arms, despite the fact they still very much feel as though they exist.

I’m convinced that VR experiences are going to transform not just games, but interaction with computers in general, and I’ve been experimenting with some ideas I have about how to create truly useful VR experiences.

As I was working to implement one of my ideas, it occurred to me that I might be able to use the orientation sensors in the Android Wear device I was wearing.  Why not use them as input into the VR experience I was creating?  What if I could bring part of my body from the real world into the VR world?  How about an arm?

I decided to try to find out, and this was the answer:

The experience is nowhere near good enough for games.  But I don’t care about games.  I want to create genuinely useful VR experiences for interacting with computers in general, and I think this is good enough.  I can point to objects, and have them light up.  I can wear smart watches on both wrists (because I really am that cool) and have two arms available in the VR world. 

By tapping and swiping on the wearable screens I can activate in-world functionality, without being taken out of it.  It sure beats sliding a magnet on the side of my face, because it is my arm I am seeing moving in the virtual world.

In the rest of this article I’m going to describe some of technical challenges behind implementing this, how I overcame them, and some of the resources I used on the way.

The tools

This is part of my workspace: Android Studio on the left, Unity on the top-right and MonoDevelop on the bottom-left:

my workspace

I had many reference browser windows open on other screens (obviously), and creating this solution required me being very comfortable in Android, Java and C#.  I’m relatively new to Unity.

Creating a Unity Java Plugin by overriding the Google Cardboard Plugin

The Unity Android Plugin documentation describes how you can create plugins by extending the UnityPlayerActivity Java class, and I experimented with this a little.  I created an Android Library using Android Studio, and implemented my own UnityPlayerActivity derived class.

After a little hassle, I discovered that Unity now supports the “aar” files generated when compiling libraries in Android Studio, although I found the documentation a little out of date on the matter in places.  It was simply a question of copying my generated “aar” file into Unity under Assets|Plugins|Android

image

image

When it came to a Google Cardboard Unity project, what I discovered though, is that Google had got there first.  They had created their own UnityPlayerActivity called GoogleUnityActivity.  What I needed to do was override Google’s override:

image

I included Google’s unity classes as dependencies in my library project:

image

Once I’d copied the aar file into the Unity Android Plugins folder and ran the test app, I was delighted to see my activity say “Cooey” in the log.

image

Receiving the watch’s orientation to the phone

The next step was to receive Android Wear Messages from Android Wear on the watch, containing orientation messages.

I recreated my project, this time including support for Android Wear:

image

I made the Unity activity I’d created do a little more than say “Cooey”. 

First I used the Capabilities mechanism to tell other Android Wear devices that this device (the phone) was interested in arm orientation messages:

image

… and I set it up to receive Android Wear messages and pass them over to Unity using UnitySendMessage:

image

Sending the watch’s orientation to the phone

This was simply a question of looking out for Android Wear nodes that supported the right capability, listening for orientation sensor changes, and sending Android Wear messages to the right node.  This is the watch code:

image

I did discover that some wearables don’t support the required sensors, although I imagine more modern ones will.

Using the watch’s orientation to animate a block on the screen

Inside Unity I created a cube which tweaked into a rectangle, and made it a child of the CardboardMain’s camera, so that it moved when I moved:

image

See the “Script” field on the bottom right-hand side?  I have a script called “WristController” that is attached to the “wrist” (white blob).  This is where I receive orientation messages sent from the watch, via the UnityPlayerActivity derived Java class I’d created.

I started off simply assigning the received orientation to the block’s orientation by assigning to transform.eulerAngles

image

This worked, but was super-jerky.  I went searching and discovered Lerps and Slerps for smoothly moving from one rotation to another.  My updated code:

image

Animating an arm instead of a block

I was pleased to be smoothly animating a block, but my arm doesn’t quite look like that.  It is more armish.  I went looking for a model of an arm that I could import and use instead.  I found a YouTube Unity video called ADDING ARMS by Asbjørn Thirslund, in which he explains how to to import and use a free arms model by eafg.

It was simply a question of sizing and positioning the arms properly as a child of the Cardboard main camera, and then adding the script I’d used to animate the block.

I also removed the right-hand arm, since it looked a little weird to have a zombie arm doing nothing.

image

The ArmController script you see in this screen capture has the same contents as the WristController I’d used to move the block.

Final Thoughts

There is enough of a lag to make this technique impractical for games, but not enough to make it impractical for the kinds of non-game experiences I have in mind. 

I’d also need to add calibration, since the watch may be pointing in any direction initially – if I assume it always starts straight out, that would be good enough.  Detecting where the arm is pointing shouldn’t be too hard, since the cardboard code already does gaze detection – so many possibilities, but so little time for side-projects such as this!

This has been a fun interlude on my way to creating what I hope to be a genuinely useful VR experience based around browsing books your friends have read … more on that later.

31Mar/133

Evernote integration with Pebble

Pebble

I signed up for the Pebble Kickstarter for a couple of reasons.

Firstly I actually wanted the watch – a waterproof watch that I could use to receive notifications without having to get the phone out of my pocket, and which I could use to control podcasts, audiobooks and music.

Secondly I wanted to be able to program it.  The full SDK is not yet available, so I can’t write programs that run on the watch, but it is possible to write Android apps that communicate with the watch.

I’ve spent some time today writing a small app that integrates the Pebble device with Evernote.

Evernote

One of the great things about Evernote is that it is very simple and easy to use at one level, but as your usage becomes more sophisticated, and your requirements deepen, you’ll find that it is much more powerful and capable than it first appears.

An example of this is that notes you create can optionally contain geolocation information, and you can view your notes on a map to see which ones you created on a particular vacation or business trip.

The idea

Vacation Location

I thought I’d use this Evernote Note geolocation capability to create an app that reminds you when you get close to the location associated with certain notes.

ColosseumAtEvening

Imagine going on vacation and creating a series of notes in Evernote for particular locations that you want to visit.  Whenever you get close to one of those locations your Pebble buzzes and you are reminded to visit it.

Evernote Food graphicRestaurant Reminder

Or imagine using Evernote Food to remember restaurants you want to visit, and your watch buzzing with a reminder if you happen to be close to one of those restaurants when it is meal time

I decided to implement it by having my app look for notes that have geolocation information (lat/lon) and are tagged with a particular tag (“pebble”).

When the app starts, it looks for those notes and uses the Android proximity service to set up proximity alerts for the note locations.  Then when the app receives the proximity alert it sends the note title as notification to the watch and could launch that note in Evernote.

image

The app

 

photo

This code uses the raw Evernote API, however there is an excellent Evernote Android SDK available, and if you are building an Android app that uses Evernote you should definitely check it out.

package com.everpebble;

import android.app.Activity;
import android.app.PendingIntent;
import android.content.*;
import android.location.LocationManager;
import android.os.Bundle;
import android.util.Log;
import com.evernote.edam.notestore.*;
import com.evernote.edam.type.*;
import com.evernote.edam.userstore.UserStore;
import com.evernote.thrift.protocol.TBinaryProtocol;
import com.evernote.thrift.transport.THttpClient;
import org.json.*;

import java.util.HashMap;
import java.util.Map;

public class Main extends Activity {

  private static final String evernoteHost = "sandbox.evernote.com";
  private static final String userStoreUrl = "https://" + evernoteHost + "/edam/user";
  private static final String TAG = "EverPebble";

  private static final String PROX_ALERT_INTENT = "com.everpebble.ProximityAlert";

  // Available from https://sandbox.evernote.com/api/DeveloperToken.action
  private static final String authToken = "TBS";

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    setContentView(R.layout.main);

    // Register for proximity events off the UI thread
    new Thread(new Runnable() {
      @Override
      public void run() {
        registerEvernoteProximityNotes();
      }
    }).start();
  }

  /**
   * Finds all notes tagged with a particular tag that have geolocation information, and registers for
   * Android proximity alerts when the phone gets close to those note locations
   */
  private void registerEvernoteProximityNotes() {
    THttpClient userStoreTrans;
    try {
      // Connect to Evernote to discover the note store URL
      userStoreTrans = new THttpClient(userStoreUrl);
      TBinaryProtocol userStoreProt = new TBinaryProtocol(userStoreTrans);
      UserStore.Client userStore = new UserStore.Client(userStoreProt, userStoreProt);
      String notestoreUrl = userStore.getNoteStoreUrl(authToken);

      // Set up the NoteStore client
      THttpClient noteStoreTrans = new THttpClient(notestoreUrl);
      TBinaryProtocol noteStoreProt = new TBinaryProtocol(noteStoreTrans);
      NoteStore.Client noteStore = new NoteStore.Client(noteStoreProt, noteStoreProt);

      // Find notes Tagged with "pebble" that have geolocation information
      NoteFilter noteFilter = new NoteFilter();
      noteFilter.setWords("tag:pebble latitude:* longitude:*");
      NoteList result = noteStore.findNotes(authToken, noteFilter, 0, 50);
      Log.i(TAG, "Found: " + result.getTotalNotes());

      // Register for proximity alerts
      LocationManager locationManager = (LocationManager) getSystemService(Context.LOCATION_SERVICE);
      for(Note note : result.getNotes()) {
        NoteAttributes attributes = note.getAttributes();
        if(attributes != null) {
          Intent intent = new Intent(PROX_ALERT_INTENT);
          
          // Save the note title and guid in the intent so that when we are close
          // we can send the title to the Pebble
          intent.putExtra("Title", note.getTitle());
          intent.putExtra("Guid", note.getGuid());
          PendingIntent proximityIntent = PendingIntent.getBroadcast(this, 0, intent, 0);
          
          // We want to be notified when we are within 100 meters of the location
          double lat =attributes.getLatitude();
          double lon = attributes.getLongitude();
          locationManager.addProximityAlert(lat, lon,  100, -1, proximityIntent);
        }
      }

      // Ensure we receive the event intents
      IntentFilter filter = new IntentFilter(PROX_ALERT_INTENT);
      registerReceiver(new ProximityIntentReceiver(), filter);

    } catch (Exception e) {
      e.printStackTrace();
    }
  }

  /**
   * Sends alerts to the pebble phone when entering within range of a note's geolocation
   */
  public class ProximityIntentReceiver extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
      Log.d(TAG, "Received proximity alert");
      boolean entering = intent.getBooleanExtra(LocationManager.KEY_PROXIMITY_ENTERING, false);

      final String title = intent.getStringExtra("Title");

      if(entering && title != null) {
        new Thread(new Runnable() {
          @Override
          public void run() {
            sendAlertToPebble("Evernote", title + " is nearby");
          }
        }).start();
      }
    }
  }


  public void sendAlertToPebble(String title, String body) {
    Log.d(TAG, "sendAlertToPebble invoked");
    final Intent i = new Intent("com.getpebble.action.SEND_NOTIFICATION");

    final Map<String, Object> data = new HashMap<String, Object>();
    data.put("title", title);
    data.put("body", body);
    final JSONObject jsonData = new JSONObject(data);
    final String notificationData = new JSONArray().put(jsonData).toString();

    i.putExtra("messageType", "PEBBLE_ALERT");
    i.putExtra("sender", "EverPebble");
    i.putExtra("notificationData", notificationData);

    Log.d(TAG, "About to send a modal alert to Pebble: " + notificationData);
    sendBroadcast(i);
  }
}

Summary

It is very early days for the Pebble, but its been great fun using it, and even more fun programming to it by integrating it with Evernote

Disclaimer: I do work for Evernote, but this blog is entirely my personal private work.

26May/1123

How to: Windows 7 incoming VPN from Android

A couple of weeks ago I tried to get my HTC Incredible S, running Gingerbread to connect via VPN to my Windows 7 box.  I failed, but today I succeeded.  This is how.

I created the incoming VPN on Windows 7 by following instructions I found on the web:

Open the Network and Sharing Center

image

Click on Change Adapter Settings

image

Press Alt-F, and select New Incoming Connection

image

Configure the incoming connection

imageimageimage

Change the IPV4 properties to specify a DHCP range

Make sure the range you specify is on your LAN, and out of the range served by your DHCP server (you can updated your DHCP server to exclude this range).

image

Check the firewall

I went to Advanced Settings under Windows Firewall, clicked on Inbound Rules, clicked on the Local Port column to sort by that column, and then verified that the line for port 1723 was set up and enabled (Routing and Remote Access (PPTP-In).

image

Set up port forwarding

I then went to my Router configuration, and set up port forwarding, so that external connections to port 1723 were forwarded to the machine I just configured (which is always allocated the same IP address by the router):

image

Now my Android device can connect to my internal VPN using the default settings.  You’ll need to know your external IP address.  You can find it using tools like http://www.whatismyip.com/ but it may change.  I have mine set up using http://www.dyndns.com/

Filed under: Android 23 Comments
18Jul/101

Tilt to turn pages in Kindle Android: Nope!

I bought an HTC Desire Android mobile phone a couple of months ago.

When I first got it, there were two applications that were missing that I really really wanted: Audible and Kindle.

image

Both were recently released, and despite this, my life is in fact not now complete.  There must be some other app for that.

I’ve been thinking about what I could develop.

As I read using the Kindle app the screen becomes smudged with my tapping the screen to change the page, and I thought it would be nice if I could change pages by tilting the phone to the right in a quick nudge.

I defied the temptation to start coding, and did some research.  It looks like it isn’t possible to do what I want using the public SDK, and to save myself the pain of rediscovering why in six months, when I have forgotten, here are some notes.

For this to work I’d need three things: A background process (called a service on Android), a way for it to detect when the phone has been tilted, and a way for it to tell the Kindle App to change the page, when it is running.

Creating a background process is pretty straightforward, and there is a sensor API that can be used to detect when the phone has been tilted.

Telling Kindle to change the page isn’t so easy.  I thought I’d found it when I discovered the IWindowManager.injectPointerEvent method, but I quickly found that this was not part of the documented Android API, and using it would be very very bad.

I was left with trying to find out whether the Kindle App itself had any way for other apps to tell it to change page.  The standard way of doing this would be to expose an Intent which other apps could invoke.

Unfortunately by dumping the Kindle App’s manifest from its installation (APK) file, it looks as though the only intents exposed are the standard MAIN, LAUNCHER, VIEW, DEFAULT and BROWSABLE intents.

Game over, unless I’m willing to go beyond the public API, which I’m not, right now at least.  Unless anyone else has any ideas?