Wednesday, July 23, 2014

What every engineer should know about Gradle when using Android Studio

Having used Android Studio since it was version 0.3, I've dealt with the growing pains of the integration of the new build system, Gradle, for quite a bit of time.  It used to be that you could create Android Studio projects without using Gradle -- now you have no choice but to use the build system.  However, thanks to recent improvements dating back to recent releases since February 2014, you also no longer have to modify the Gradle file directly to accommodate any additional library files since Android Studio tries to handle things for you.

Regardless, here are a few things that I thought are useful to know if you're using Gradle and Android Studio:

1. There are multiple build.gradle files.  When you create a new project, Android Studio creates two build.gradle files. The first is the parent that encompasses all modules in your project, while the 2nd is the one that pertains directly to the app. You will in most cases be dealing with this second build.gradle file. You do not really need to make any changes to the 1st one, which is blank by default.

2. Adding JAR files: When you drop a file into the designated libs/ directory, the changes are usually updated in the local build.gradle file.  You can see the changes reflected in the dependencies section.  For instance, if you drop the android-async-http-1.4.3.jar, it gets explicitly added to the dependency tree, even though the first statement should suffice:

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile files('libs/android-async-http-1.4.3.jar')
}
The addition of the 2nd is redundant because of the 1st statement -- however, if you add a library file outside the libs/ directory, presumably it will be automatically added to this list.

3. Removing JAR files: When you delete the JAR file, Android Studio currently not does handle removal of these .JAR files in the build.gradle file.  If you intend to remove one, you should check the build.gradle to see if you also need to delete the line that deals with importing the JAR file.

4. Adding library dependencies: If you're trying to add another library such as the Facebook SDK, which will usually reside outside the scope of your main app, you need to update the settings.gradle in the parent directory to reference this module.  For instance, if you copy the contents into a Libraries/facebook directory, you need to have the 2nd extra include statement:
include ':app'
include ":Libraries:facebook"
If your app intends to depend on this SDK, you also need to reference it inside the build.gradle of your app:
dependencies {
    compile project(':Libraries:facebook')

If you try to use the compile project statement without the changes to settings.gradle, you will likely see an error about "Project with path :Libraries:facebook could not be found in project :app".  If you want a working example, check out Jonathan Azoff's repo: https://github.com/azoff/codepath-twitter-client

For more context, see the section on Multi project setup at http://tools.android.com/tech-docs/new-build-system/user-guide.

5. Use the Gradle wrapper option. When you select the option, you are effectively generating a few files that enable other people who check out your project not to have to install Gradle themselves once you've generated the files needed to bootstrap the process.  It is the recommended that you check these files into your source code to allow others to be able to compile your program successfully.
  
  gradlew
  gradlew.bat
  gradle/wrapper/
    gradle-wrapper.jar
    gradle-wrapper.properties

The exact Gradle version that will be downloaded and used to build your app is defined in the gradle/wrapper/gradle-wrapper.properties file. The downloaded version is usually cached in ~/.gradle/wrapper/dists.

6. Unsure about whether Android Studio is creating build issues?  If you'd like to see if the gradle build succeeds, you can always run things by executing Gradle at the command line.  Within your project, you can type "bash gradlew assembleDebug" or "gradlew.bat assembleDebug", which will attempt to compile and build your Android app.

7. Seeing Duplicate files copied in APK errors?  Double-check your build.gradle files in both your main Android app and your libraries.  Your main app should be using the 'android' plug-in, while your libraries should be using the android-library plugin.

I made the mistake of trying to define all libraries with the 'android' plugin, triggering this issue possibly since Gradle didn't know how to resolve duplicate AndroidManifest.xml and other shared library files.

For instance, your main app should have the android plugin:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:0.12.+'
    }
}
apply plugin: 'android'

Your other libraries should be using the android-library plugin:
buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:0.12.+'
    }
}
apply plugin: 'android-library'
You can also take a look at the explanation at http://tools.android.com/tech-docs/new-build-system/user-guide that covers the different between a project and a library project.  A project will generate an Android Application Package (APK) file, while a library project will generate an Android ARchive (AAR) file.  Since you can only have one APK file, it makes sense that you're only allowed to define one project.  Android Studio currently doesn't prevent you from making this mistake however.

8. New versions of Gradle may require the Android Gradle plugin to be updated too.  

When you add the "apply plugin: 'android'", you are in fact instructing Gradle to use the Android Gradle plugin, which Google maintains at https://android.googlesource.com/platform/tools/base/+/master/build-system/gradle/src/main/groovy/com/android/build/gradle/internal/).   This plugin helps inform Gradle where to find your Android SDK, which is usually defined in local.properties.

The evolving nature of Gradle and the Android Gradle plugin means that the versions you need may need to be updated. You should always check the gradle/wrapper/gradle-wrapper.properties and the build.gradle file to verify that the versions you are attempting to use are compatible.  Check the Android tools link http://tools.android.com/tech-docs/new-build-system to verify.

9. Seeing cryptic "failed to find Build Tools revision" errors?


Double-check the Android SDK Build Tools version specified in your buildToolsVersion reflects the one that's installed i Android SDK Manager.  You may need to tweak your buildToolsVersion to be updated to one that you have installed.    Similarly, you may also need to do the same for the compileSdkVersion definition.

android {
    buildToolsVersion "19.1.0"

10. Still confused?  Read the writeup of new Android Gradle plugin build system.

Friday, July 18, 2014

Where Google keeps the Gradle source code..

Go to:

https://android.googlesource.com/platform/tools/base/

Instead of downloading the entire Android source code base, you can click on one of the branches and click on the 'tgz' link to download a snapshot of the tree.


Sunday, July 13, 2014

How to read serial flash chips


Serial flash have become so prevalent in all electronic devices, replacing parallel based ones. Wondering how you can read off the data from them?  The UM232H USB device can be used for this purpose.  It can be purchased from Mouser.com or DigiKey.com for $20.  In order to power the device, you need to wire things up based on Section 7.2 of the spec -- the VIO/3v3 and 5VO/USB pins should be shorted accordingly:


To verify the USB device is powered, an LED light should be lighted once you plug a Mini-USB cable to the connector.   If you're using OS X, you can check that the device is connected on USB (system_profiler SPUSBDataType).

          Product ID: 0x6014
          Vendor ID: 0x0403  (Future Technology Devices International Limited)
          Version:  9.00
          Serial Number: FTVRYH0J
          Speed: Up to 480 Mb/sec
          Manufacturer: FTDI
          Location ID: 0x14500000 / 10
          Current Available (mA): 500
          Current Required (mA): 90

In addition to the UM232H device, I ordered the Winbond W25Q serial flash chips from DigiKey, an SOIC to DIP adapter, and a small breadboard with wires to connect the chips.   I originally ordered the Adesco AT45 SPI flash chips from Mouser, but these chips were too wide for the 150mil SOIC to DIP adapter I ordered.  In order to make these chips work, I had to order the 208mil SOIC to DIP adapter, which is currently taking 2-3 weeks from China to arrive.

The pyftdi project provides Python bindings to interface with the UM232H controller, which in turn can send SPI commands to serial flash chips.  What the pyftdi project does is enable MPSSE mode on the UM232H device, which in turn enables pins AD0-AD4 to act as SCK, DI, DO, and CS select signals.   Note the MPSSE column in the UM232H manual:


In order to discern what pins 13-16, I had to look at the circuit schematic, which shows that ADBUS0-ADBUS3 corresponds to the AD0-AD3 pins of the UM232H device.    Note that the UM232H spec also has things listed according to the target device relative to the UM232H device. For instance, TDI/DO, means the wires should run to the DI input of the SPI Flash and match the data output of the UM232H device.  I've included the schematics from the UM232H and the W25Q serial flash chip below:



Using the pyftdi project, I was able to issue the JEDEC command to get the manufacturer and device information back from the chip, which helped confirm that everything was working.  The 0x403 and 0x6014 correspond to the vendor and product ID of the UM232H USB device:

from serialflash import SerialFlashManager
from pyftdi.pyftdi.spi import SpiController

flash = SerialFlashManager.get_flash_device(0x403, 0x6014)
print flash

I had to make a few tweaks to ensure that the JEDEC information could be correctly.  The setup is currently using somewhat long wires between devices, so my goal is to reduce this length since it takes a couple of tries before the device returns the manufacturer ID correctly.

Saturday, June 28, 2014

Android LG G Watch: First Impressions

My co-worker Ruchi gave me the LG G Watch to try this weekend, and I immediately thought it to be way too difficult for the average consumer to understand.  Since it has no power button, you have to plug it into the cradle to enable.  Also, for developers who didn't attend the I/O conference, you have to signup to join the Google Wearables Group before you can be whitelisted to download the Android Wear app.  Figuring that I had to wait at least an hour, I decided to try to use the watch later.

A few hours later, the phone defaulted into some type of retail mode, where it demo's the various functionality of the phone.  To exit this mode, I had to factory reset the watch to retry the pairing process.  The Android Wear app itself is buggy: it complains that you have to update your Google Play Services even when you have the latest version.  The smartwatch eventually showed up for pairing on my tablet, but then it failed to find Internet connectivity even though the WiFi was fully functioning.  Now I'm stuck in a state where the tablet can see the watch for pairing, but fails to do anything after the next step.

This launch for the Android wearables seems to have been rushed for the Google I/O conference. Here's hoping Google's actual release next week will actual polish all these issues.  For now, it feels like a half-baked product.

How to Factory Reset Android LG Wearable Watch

The watch has no buttons, so you have to plug it into the cradle to enable.

If your phone gets into "retail mode", you apparently have to push the small pin on the back of watch and swipe right on the screen until you get to the Factory Reset icon.


Wednesday, June 18, 2014

Building your own Python version for an easier debugging experience.

One of the major issues in using the stock Python distribution (i.e. Ubuntu) is that it's compiled with a bunch of optimizations in the final binary. You can install the debug symbols (via the python-dbg package), but if you want to be able to troubleshoot any C-based extensions, it's likely you're going to want to be able to examine what lines actually triggered segmentation dumps by using the core file.   In addition, if you can also break into a running Python process or want to inspect the memory structures to see what's actually being allocated, you unfortunately can't do this type of debugging with the standard build.

What if you simply wanted to have a version of Python that could be used for debugging? What if you still wanted to keep the standard version too? Here's the steps that I took to accomplish this goal.  To make sure the bzip2 and sqlite3 modules were installed in the Python distribution, I had to install the development packages for them.  I also had to override the default C compile flags by using OPT="" and CFLAGS="-O1 -g" (the -g flag produces debugging symbols)

cd /somewhere/you/want/source/code/
sudo apt-get source python
sudo apt-get install libbz2-dev
sudo apt-get install libsqlite3-dev
OPT="" CFLAGS="-O1 -g" ./configure --prefix=/home/rhu/.virtualenvs/pydev --with-pydebug
make install

The next step is to install the virtualenv package from source. Normally you don't need to do this part but because we are using a custom Python binary, we need to use it for the compile step.  You can download the latest version from http://pypi.python.org/packages/source/v/virtualenv.  I decided to store it in the .virtualenvs directory that is normally created by the virtualenvwrapper package, which provides shortcuts to activate and deactivate Python virtual environments.

tar -zxvf virtualenv-1.x.x.tar.gz
cd virtualenv-1.x.x/
~//home/rhu/.virtualenvs/pydev/bin/python setup.py install

You'll need to activate the virtual environment by running the activate script. If you have the virtualenvwrapper package setup, you can also just type "workon pydev".

Note that all your Python packages will need to be recompiled.   They should inherit the same compiler flags used for building your Python package, so you should see "-O1 -g" compiler optimizations being used.  Note that I used the --with-pydebug flag, which will do some reference debugging and make your existing libraries incompatible.  By doing all of this work inside a virtual environment, you can avoid library conflicts with other installations.

(Note: using -O0 caused some code such as the librabbitmq to fail to compile for some reason, so I opted to keep the minimal amount of optimizations enabled.  If someone figures out what the minimal compiler optimizations to use that would allow the -O0 flag to be used, please let me know!  )