Archiving Gmail (Part 3)

Time for a little aside.  In the last post I showed you were we were headed with a Google apps script web application that will archive or forward gmail messages in bulk.  Running this script can take a while if there are a lot of emails and Google apps scripts don't have a very good method for showing that they are running.

While I wanted a progress bar, I couldn't find anything that would easily get me there (although if you read Solution 2 below you could back your way into one but it would be cumbersome at best).

I settled on showing and hiding a spinning gif to indicate to the user the script is still running.  I found two ways to do this, but only one (the second) works with the menus I wanted to use.

Solution 1:

This solution does NOT work with menus.  Basically the idea is you add a normal server handler to a button (for example) then add an additional client handler to the MouseUp event. Click to see it in action.

Here is the simple code for that demo:

// Script-as-app template.
function doGet() {
  var app = UiApp.createApplication();

  var label = app.createLabel('click the button.')
                 .setId('statusLabel')
                 .setVisible(true);
  
  app.add(label);
  var spinner = app.createImage('https://5079980847011989849-a-1802744773732722657'+
            '-s-sites.googlegroups.com/site/scriptsexamples/ProgressSpinner.gif')
            .setVisible(false)
            .setId('spinner');
  app.add(spinner);

  
  var myButtonHandler=app.createServerHandler('myButtonLongRunningFunction');
  
  var myButton=app.createButton("Click to Test", myButtonHandler);
    var loadSpinner = app.createClientHandler()
            .forTargets(spinner)
            .setVisible(true).forTargets(label).setText('Running...');

  myButton.addMouseUpHandler(loadSpinner)
  app.add(myButton);
  return app;
  
}

function myButtonLongRunningFunction(e){

  Utilities.sleep(5000);
  
  var app=UiApp.getActiveApplication();
  app.getElementById('statusLabel').setText('Done.');
  app.getElementById('spinner').setVisible(false);
  return app;
}

Solution 2:

As I stated, the above method will NOT work if you use a menu.  Why? Because you can't assign both a client handler and server handler to a menuItem.   So how can we accomplish the same thing using only a server handler?  By tricking the script into running TWO server handlers asynchronously.  Click here to see this in action

Here are the basic steps:

  1. Create your menu
  2. Create an object like a textbox to hold the value of the menu item clicked
  3. Create a checkbox with a ValueChangeHandler.  this handler will check the textbox for the command that was requested, and perform some long running action.
  4. create a menu handler for each menuitem.  these menu handlers will show the progress indicator, set the value in the textbox, then toggle the checkbox value to trigger the ValueChangeHandler created above.

Here is the code for example two:

 

// Script-as-app template.
function doGet() {
  var app = UiApp.createApplication();

  //Create menu
      var menuBar = app.createMenuBar();//For horizontal menubar
    var handler1 = app.createServerHandler('myMenu1Handler');
    var handler2 = app.createServerHandler('myMenu2Handler');
  
  
    var menuItem0 = app.createMenuItem('Item 1', handler1);
    var menuItem1 = app.createMenuItem('Item 2', handler2);
      var separator0 = app.createMenuItemSeparator();
    var separator1 = app.createMenuItemSeparator();
    menuBar.addSeparator(separator0).addItem(menuItem0).addSeparator(separator1)
    .addItem(menuItem1);
  app.add(menuBar);
  
  //Create the control checkbox trigger and textbox
  var myControlTextBox=app.createTextBox().setId('controlTextBox').setName('controlTextBox').setVisible(false);  
  var controlHandler=app.createServerHandler('myControlHandler');
  
  var myControlCheckBox=app.createCheckBox().setId('controlCheckBox').setName('controlCheckBox').addValueChangeHandler(controlHandler).setVisible(false);

  controlHandler.addCallbackElement(myControlTextBox);
  controlHandler.addCallbackElement(myControlCheckBox);

  app.add(myControlTextBox);
  app.add(myControlCheckBox);

  //create the spinner we will use for progress
    var spinner = app.createImage('https://5079980847011989849-a-1802744773732722657'+
            '-s-sites.googlegroups.com/site/scriptsexamples/ProgressSpinner.gif')
            .setVisible(false)
            .setId('spinner');
  app.add(spinner);
  var resultsLabel=app.createLabel().setVisible(false).setId('resultsLabel');
  app.add(resultsLabel);
  return app;
}

function myControlHandler(e){
  Logger.log("control called");
  var app=UiApp.getActiveApplication();
  if(e.parameter.controlCheckBox=='true'){
    if(e.parameter.controlTextBox=='Menu1'){
      
      Utilities.sleep(3000);
      
      app.getElementById('spinner').setVisible(false);
      app.getElementById('resultsLabel').setText('Menu 1 was clicked').setVisible(true);
      //reset control box
      var controlCheck=app.getElementById('controlCheckBox');
      controlCheck.setValue(false,false);
    } else if(e.parameter.controlTextBox=='Menu2'){
      Utilities.sleep(3000);
      app.getElementById('spinner').setVisible(false);
      app.getElementById('resultsLabel').setText('Menu 2 was clicked').setVisible(true);
      //reset control box
      var controlCheck=app.getElementById('controlCheckBox');
      controlCheck.setValue(false,false);
    } else {
      Logger.log("controlTextBox value: " + e.parameter.controlTextBox);
    }
  } else {
    //Do nothing - this is result of reset from above.
  }
  return app;
}
function myMenu1Handler(e) {
  var app = UiApp.getActiveApplication();
  //show the spinner
  app.getElementById('spinner').setVisible(true);
  app.getElementById('controlTextBox').setText('Menu1');
  app.getElementById('resultsLabel').setVisible(false);
  var controlCheck=app.getElementById('controlCheckBox');
  controlCheck.setValue(true,true);
  return app;
}
function myMenu2Handler(e) {
  var app = UiApp.getActiveApplication();
  //show the spinner
  app.getElementById('spinner').setVisible(true);
  app.getElementById('controlTextBox').setText('Menu2');
  app.getElementById('resultsLabel').setVisible(false);
   var controlCheck=app.getElementById('controlCheckBox');
  controlCheck.setValue(true,true);
  return app;
}

Since I decided that I liked the look of the menu better, I went with option 2.

to be continued....

Archiving Gmail (Part 2)

In my last post, I addressed one of the issues I had with a script I had found that archives Gmail messages to PDF files and how to fix the issue of it creating needless temporary files that count toward your daily quota.

The remaining issues I had I addressed by moving the entire script from a Google spreadsheet to a Google web app script.  This was the original script UI:

Screen Shot 2013-11-20 at 8.38.27 AM.png

And here is the new UI:

DemoApp.png

As you can see, I it has a few more options than the original.  The gist of it is

  • you select the Gmail Label you want to use as a filter, you optionally select a label you want to ignore (You could create two labels for example 'Archive' and 'Processed').  You can also add your own custom filter to include or exclude additional messages.  
  • Choose if you want to save the attachments (If you are archiving the emails to PDF).
  • Optionally you can have it remove the first Label (e.g. 'Archive') and Optionally add a new label (e.g. 'Processed').  Both of these are helpful when you have a large number of emails to process and may bump against your daily Quotas for creating or forwarding mail.
  • If you want to Archive them, you can archive them to PDF with the attachments saved as separate files, or you can save them to EML files with the attachments embedded in the original email.  You can change the naming convention to a few different formats (This makes it easier to sort them later so you could specify From-Date-Id instead for example)
  • Or you can bulk forward them to some specified recipient(s).

If you want to see the finished product here is the link.  You will need to authorize it to access your mailbox etc since obviously it is processing mail.

 

So where to begin?

First you need to setup your Google Drive to write the scripts.  

  • In Drive, Click Create -> Connect More Apps
Screen Shot 2013-11-20 at 8.56.36 AM.png
  • Search for "script" and connect to Google Apps Script
Screen Shot 2013-11-20 at 8.59.24 AM.png
  • Now create your first script by creating a new script.  The type you want will be Script as Web App which will be available after you choose Script from the below image.
Screen Shot 2013-11-20 at 9.01.05 AM.png
  • You will now have your development environment all set to start scripting!
Screen Shot 2013-11-20 at 9.04.23 AM.png

Well, this is getting rather long, so in the next post I will just dump the code for the app I wrote and walk you through it.  Click Here to see Part 3

Archiving Google Mail To Files

I was recently tasked with extracting 2-300 gmail messages including attachments.  The requirements seemed simple enough:

  • Every message must include original meta data (from/to/cc/bcc/date etc)
  • all attachments must be saved in original format

Obviously one way would have been to sync the messages to outlook, create a folder in outlook, copy and/or move the messages to that folder, then export it to PST.  

But it seemed like there should be a 'native' gmail way to do it.  So I started poking around and found a lot of Google Apps Scripts had been written that would bulk forward or archive messages.   This one by Marcello Scacchetti caught my eye. 

But as I started using it, I realized there were a few aspects I didn't really like:

  1. It doesn't save the meta data
  2. It creates temp files in Google Drive
  3. Hard coded naming conventions
  4. Hard coded to PDF output.
  5. It hard coded labels
  6. It was built in a spreadsheet.

To keep this post short, I will break this into several posts and only address one issue here.  Since #1 is easy enough for the average bear to fix themselves so I won't bother addressing that.  The first real problem was with #2. Here we see the offending code:

 // Create the message PDF inside the new folder
var htmlBodyFile = newFolder.createFile('body.html', messageBody, "text/html");
var pdfBlob = htmlBodyFile.getAs('application/pdf');
pdfBlob.setName(newFolderName + ".pdf");
newFolder.createFile(pdfBlob);
htmlBodyFile.setTrashed(true);

Before it creates the pdf file, it creates an HTML file.  Why is this a problem? Well in scripts (particularly individual not paid corporate gmail accounts) there are quotas on how many files you can create via script per day - 250 as of the time of this writing.  That means that you could only archive 125 emails with this, and then only if none of them had attachments.  Luckily this is easy enough to fix:

var output = HtmlService.createHtmlOutput(messageBody)
var pdfBlob=output.getAs('application/pdf');
pdfBlob.setName(newFolderName + ".pdf");
newFolder.createFile(pdfBlob);
Here we use the HTMLService to create an object rather than a file. Easy fix and doubles the number of emails we can process per day.

I'll post some solutions to my other issues with the script in the next few days.