Damian Mehers' Blog Evernote and Wearable devices. All opinions my own.

11Jan/118

Generating Silverlight / Windows Phone compatible Thrift proxies

Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml. http://thrift.apache.org/

There are two reasons for this post:

  • to float some ideas for C#changes to the Thrift development community.
  • to remind myself in the future what I need to do to download, modify and build the Windows Thrift compiler, and

The idea is to maintain backwards compatibility, whilst at the same time generating Silverlight (and consequently Windows Phone 7) compatible proxies.

There are two sets of changes.  One to the C# code generator (t_cpp_generator.cpp), and the other to the runtime files (mainly THttpClient.cs).

Downloading and building the thrift compiler

To try out these changes on Windows, first install the latest version of Cygwin from http://cygwin.com/, then install the various packages as described here: http://wiki.apache.org/thrift/ThriftInstallationWin32

You’ll also need to install Subversion.

Next pop open the Cygwin shell, and fetch the latest source, as described here: http://thrift.apache.org/download/

Finally, build the source you downloaded, again as described here:

http://wiki.apache.org/thrift/ThriftInstallationWin32

In my environment I build using:

image

And then I run the new thrift compiler:

image

Code Generator changes

The code generator (t_cpp_generator.cpp) changes involve generating two additional methods for each “standard” method: a Begin_… method and an End_… method.

Whereas previously there might just have been a method such as this:

   1: SyncState getSyncState(string authenticationToken);

Now, two additional methods get generated:

   1: #if SILVERLIGHT

   2: IAsyncResult Begin_getSyncState(AsyncCallback callback, object state, string authenticationToken);

   3: SyncState End_getSyncState(IAsyncResult asyncResult);

   4: #endif

The two methods allow for the asynchronous invocation of methods, using the standard .NET asynchronous invocation pattern.

In addition, the generated standard method (“getSyncState”) method is modified when building for Silverlight, to make use of the Begin… and End… methods:

   1: public SyncState getSyncState(string authenticationToken)

   2: {

   3:     #if !SILVERLIGHT

   4:     send_getSyncState(authenticationToken);

   5:     return recv_getSyncState();

   6:  

   7: #else

   8:     var asyncResult = Begin_getSyncState(null, null, authenticationToken);

   9:     return End_getSyncState(asyncResult);

  10:  

  11: #endif

  12: }

As you can see, when not running Silverlight the standard code path is invoked, but when running Silverlight the asynchronous methods are invoked (the End… method blocks the current thread until the Begin… method completes).  This is not something you should be doing on the UI thread.

The generated Begin… and End… methods are pretty thin:

   1: public IAsyncResult Begin_getSyncState(AsyncCallback callback, object state, string authenticationToken)

   2: {

   3:     return send_getSyncState(callback, state, authenticationToken);

   4: }

   5:  

   6: public SyncState End_getSyncState(IAsyncResult asyncResult)

   7: {

   8:     oprot_.Transport.EndFlush(asyncResult);

   9:     return recv_getSyncState();

  10: }

Note the call to EndFlush above – this is one of the changes made to the runtime.  The other is invoked by the generated send_getSyncState method:

   1: #if SILVERLIGHT

   2: public IAsyncResult send_getSyncState(AsyncCallback callback, object state, string authenticationToken)

   3: #else

   4: public void send_getSyncState(string authenticationToken)

   5: #endif

   6: {

   7:     oprot_.WriteMessageBegin(new TMessage("getSyncState", TMessageType.Call, seqid_));

   8:     getSyncState_args args = new getSyncState_args();

   9:     args.AuthenticationToken = authenticationToken;

  10:     args.Write(oprot_);

  11:     oprot_.WriteMessageEnd();

  12: #if SILVERLIGHT

  13:     return oprot_.Transport.BeginFlush(callback, state);

  14: #else

  15:     oprot_.Transport.Flush();

  16: #endif

  17: }

The generated recv_getSyncState() has not changed.

The changes in the generated code boil down to asynchronous invocations at the transport layer (BeginFlush and EndFlush).

There are also changes to #ifdef out the Serializable attribute for generated structs, since this is not supported by Silverlight.

Runtime changes

There are minor tweaks required to THashSet, TProtocol and TBinaryProtocol because Silverlight does not support the full .NET framework API.

The main change is to TTransport.cs to introduce the BeginFlush and EndFlush methods shown above, and then to THttpClient.cs to actually implement these methods.

The existing Flush method is #ifdefed out when building using Silverlight, because it makes synchronous calls which are not supported by Silverlight.

Instead, the BeginFlush builds a request and then invokes the HttpWebRequest.BeginGetRequestStream method, passing a local GetRequestStreamCallback method as the callback.

The GetRequestStreamCallback method is invoked once the runtime has the request stream.  It then writes out the data and invokes the HttpWebRequest.BeginGetResponse method, passing a local GetResponseCallback method as callback.

The GetResponseCallback notifies the original caller (in the generated code) that the request has now completed.

The EndFlush method waits for the corresponding BeginFlush method to complete, and if there was an exception thrown at any point, it raises the corresponding exception.

The changes

The changes files are here.  If there is interest and these changes make sense, I’ll submit a patch.

Comments (8) Trackbacks (0)
  1. I’m new to Thrift. In fact, I only heard about it a few days ago when I was looking into the Evernote.com API. It uses Thrift, and currently the API ships with a C# project, but it won’t work with Silverlight since it is sychronous only. I think this would be a great patch to add into the code generator. And perhaps an example post showing us newbies how to make use of it.

  2. Hi,
    I have replaced the cs files, but I get this exception:
    The method or operation is not implemented.
    At this line:
    connection.UserAgent = “C#/THttpClient”;

    I’m trying to use the username/password login like a desktop application. Does that make the problem?
    Thanks

  3. Damian,

    are you still planning to provide the patch? Aside from some troubles I still have with it, I like your code in general. Could you please give me a mail to discuss this further?

    Thank you,
    Jens

  4. Hi Damian, thanks for this work. I’m busy investigating building a Thrift service that will be used by a Silverlight client. Without your work I wouldn’t even have considered using Thrift.

  5. You are welcome Jaco – glad its getting used by more people than just me!

  6. Hi Damian, any chance you can point me to an example of a silverlight client and C# server using HTTP as transport? Unfortunately there’s no tutorials in the source that show me how to go about this. I assume I have to make use HTTP since sockets are not supported?

  7. hey damian i was developing a metrostyle app using your api and i ran into some troubles , Firstly the code given has got some errors but i got my way around them with slight modifications but the main problem thats bugging me is that how to attach media to the note . i have done a 12 hrs research to implement this but got nowhere see to attach a media i used the following code (on the lines of php code given here http://dev.evernote.com/documentation/cloud/chapters/Creating_notes.php i dont know why u guyz provide samples in one language only)

    Data data = new Data();
    data.Body = arr; // here arr is the byte array of the image i want to send got it with 3 hrs as no // system.drawing package for wp7

    Resource io = new Resource();
    io.Mime = Class1.imp.ToString();
    io.Data = data;
    //io.Attributes.FileName = imm.ToString();
    string p=”Image/jpeg”;
    List top = new List();
    top.Add(io);
    Note newNote = new Note
    {
    NotebookGuid = defaultNotebook.Guid,

    Content = “” +
    “” +
    “some random shit ” +
    “<en-media
    // please tell me how to do this as !! " type=”p,” hash =”gash, “_ “,!! this version doesnt work
    “”,
    Title = “fuddu”};

    newNote.Resources = top;
    Note createdNote = noteStore.createNote(authToken, newNote);

    ShowMessage(“Successfully created new note with GUID: ” + createdNote.Guid);

  8. Sorry for the late response. There is an Evernote example here that shows how you can attach resources:
    https://github.com/evernote/evernote-sdk-csharp/blob/master/sample/client/SampleApp/EDAMTest.cs

    Regards,
    Damian


Leave a comment

No trackbacks yet.