I've often found that anytime you upgrade Jenkins versions, the plug-ins that accompany it often break. The other day, the scm-sync-configuration plugin that helps manage and store the XML-based configuration files stopped working after upgrading to Ubuntu 14.04 and the Jenkins package included with it. The plugin appeared to copy files to the scm-sync-configuration/ directory but never seemed to commit the changes to the Git repo. What was wrong?
It turns out that the issue was already reported since April 2014. The workaround for most people was to downgrade from Git 1.9, which comes with the Ubuntu 14.04 installs, to Git 1.8 versions. It so happens that the maven-scm-provider-git plugin on which this Jenkins plugin depends attempts to parse the output of the "git status". When the formatting change, it broke the ability to ascertain what changes had been made. The related ticket is here:
https://jira.codehaus.org/browse/SCM-738
The issue for this Maven project was fixed subsequently, but the dependent packages that the scm-sync-configuration plugin had to be updated. Furthermore, you had to check that the maven-scm-api that has this fix actually got updated to 1.9.1, since missing one upgrade dependency meant that it would stay at an older version.
The changes for this fix are here:
https://github.com/jenkinsci/scm-sync-configuration-plugin/pull/21
The other issue, which was much less obvious, was that there's a related XML file that determines what source code control systems are supported. For some reason, only Subversion (SVN) was declared, which broke tests for the Git-related ones. It wasn't so obvious until several hours of tracing through the source code and noticing that the ScmManager class only had one ScmProvider declared. Apparently this plugin uses a package called Plexus to load this information, which relies on XML configurations such as this file.
Saturday, August 23, 2014
Wednesday, August 20, 2014
Running RabbitMQ on Ubuntu 14.04?
Recently during a few test runs we noticed that our AMQP clients would often lock up when interfacing with a RabbitMQ machine running on Ubuntu 14.04 hosted on Amazon AWS. By installing tshark and capturing the network traces on the host running the AMQP client, I noticed there seemed to be issues reported as "Unknown frame type". Upon further investigation of the unknown frame packet, I noticed that we seemed to be missing the AMQP header, causing the packet to be classified as unknown by Wireshark. Normally there should have been Basic.Deliver like all the previous packets:
I noticed on the machine running on RabbitMQ that there were "xen_netfront: xennet: skb rides the rocket" error messages. It turns out that AMQP is very sensitive to missing frame headers and caused both RabbitMQ and the AMQP client to wait on each other. The broker thinks there are more messages waiting to be acknowledged, while the client sees no new messages. The reason appears to be that the missing AMQP header caused by the dropped transmit packet causes this issue:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1317811
http://www.spinics.net/lists/netdev/msg282340.html
The solution appears to be disable scatter/gather I/O on the network card, since the Xen driver may take longer to resolve:
sudo ethtool -k eth0
The patch that appears to have started contributing to these issues:
https://gitorious.org/linux-can/linux-can-next/commit/f36c374782e40a3812f729838b5b80d2ce601725
I noticed on the machine running on RabbitMQ that there were "xen_netfront: xennet: skb rides the rocket" error messages. It turns out that AMQP is very sensitive to missing frame headers and caused both RabbitMQ and the AMQP client to wait on each other. The broker thinks there are more messages waiting to be acknowledged, while the client sees no new messages. The reason appears to be that the missing AMQP header caused by the dropped transmit packet causes this issue:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1317811
http://www.spinics.net/lists/netdev/msg282340.html
The solution appears to be disable scatter/gather I/O on the network card, since the Xen driver may take longer to resolve:
sudo ethtool -k eth0
The patch that appears to have started contributing to these issues:
https://gitorious.org/linux-can/linux-can-next/commit/f36c374782e40a3812f729838b5b80d2ce601725
Tuesday, August 19, 2014
Stepping through a running Python program through gdb
The main part is that you need to have your custom Python and gdb build. The Ubuntu Python version has compiler optimizations removed, which makes it difficult to use. You'll also need to install all your package dependencies using the virtual environment that you've setup.
1. Build your own custom Python version. Instructions are here.
2. If you're using Ubuntu 14.04, you need to compile your own gdb too, since the one that comes is linked to Python 3:
a. Download the source from https://launchpad.net/ubuntu/+source/gdb/7.6.1-0ubuntu3.
b. sudo apt-get install libncurses5-dev texinfo
c. ./configure --with-python=python2 --prefix=/home/rhu/special (or wherever you intend to do make install).
d. make install
3. Locate the libpython.py from your Python install.
a. sudo /home/rhu/special/bin/gdb -p <python process>
b. python
sys.path.insert(0, "/home/rhu/projects")
import libpython
You should now be able to do py-up, py-down, py-list, py-locals to step through the various call stacks.
1. Build your own custom Python version. Instructions are here.
2. If you're using Ubuntu 14.04, you need to compile your own gdb too, since the one that comes is linked to Python 3:
a. Download the source from https://launchpad.net/ubuntu/+source/gdb/7.6.1-0ubuntu3.
b. sudo apt-get install libncurses5-dev texinfo
c. ./configure --with-python=python2 --prefix=/home/rhu/special (or wherever you intend to do make install).
d. make install
3. Locate the libpython.py from your Python install.
a. sudo /home/rhu/special/bin/gdb -p <python process>
b. python
sys.path.insert(0, "/home/rhu/projects")
import libpython
You should now be able to do py-up, py-down, py-list, py-locals to step through the various call stacks.
Solving the "had used a different Landroid/support/v4/app/FragmentActivity; during pre-verification" error
This Stack Overflow pointed to issues with including the support-v4.jar twice.
http://stackoverflow.com/questions/5561353/fragmentactivity-can-not-be-tested-via-activityinstrumentationtestcase2/5645277#5645277
The solution?
If you're using Roboelectric, the support-v4.jar will likely be specified as a compilation dependency. If you see "(Lcom/example/android/app/FragmentLayoutSupport; had used a different Landroid/support/v4/app/FragmentActivity; during pre-verification) error if it is not excluded.", it's likely it needs to be defined as part of your exclude settings.
http://stackoverflow.com/questions/5561353/fragmentactivity-can-not-be-tested-via-activityinstrumentationtestcase2/5645277#5645277
The solution?
If you're using Roboelectric, the support-v4.jar will likely be specified as a compilation dependency. If you see "(Lcom/example/android/app/FragmentLayoutSupport; had used a different Landroid/support/v4/app/FragmentActivity; during pre-verification) error if it is not excluded.", it's likely it needs to be defined as part of your exclude settings.
dependencies { compile fileTree(dir: 'libs', include: ['*.jar']) // borrowed from https://github.com/robolectric/deckard-gradle/blob/master/build.gradle androidTestCompile 'org.hamcrest:hamcrest-integration:1.1' androidTestCompile 'org.hamcrest:hamcrest-core:1.1' androidTestCompile 'org.hamcrest:hamcrest-library:1.1' // http://saltnlight5.blogspot.com/2012/10/whats-up-with-junit-and-hamcrest.html androidTestCompile('junit:junit:4.+') { exclude module: 'hamcrest-core' } androidTestCompile('org.robolectric:robolectric:2.3') { exclude module: 'classworlds' exclude module: 'commons-logging' exclude module: 'httpclient' exclude module: 'maven-artifact' exclude module: 'maven-artifact-manager' exclude module: 'maven-error-diagnostics' exclude module: 'maven-model' exclude module: 'maven-project' exclude module: 'maven-settings' exclude module: 'plexus-container-default' exclude module: 'plexus-interpolation' exclude module: 'plexus-utils' exclude module: 'support-v4' // avoid support v4 DEX conflicts exclude module: 'wagon-file' exclude module: 'wagon-http-lightweight' exclude module: 'wagon-provider-api' } }
Wednesday, July 23, 2014
What every engineer should know about Gradle when using Android Studio
Having used Android Studio since it was version 0.3, I've dealt with the growing pains of the integration of the new build system, Gradle, for quite a bit of time. It used to be that you could create Android Studio projects without using Gradle -- now you have no choice but to use the build system. However, thanks to recent improvements dating back to recent releases since February 2014, you also no longer have to modify the Gradle file directly to accommodate any additional library files since Android Studio tries to handle things for you.
Regardless, here are a few things that I thought are useful to know if you're using Gradle and Android Studio:
1. There are multiple build.gradle files. When you create a new project, Android Studio creates two build.gradle files. The first is the parent that encompasses all modules in your project, while the 2nd is the one that pertains directly to the app. You will in most cases be dealing with this second build.gradle file. You do not really need to make any changes to the 1st one, which is blank by default.
2. Adding JAR files: When you drop a file into the designated libs/ directory, the changes are usually updated in the local build.gradle file. You can see the changes reflected in the dependencies section. For instance, if you drop the android-async-http-1.4.3.jar, it gets explicitly added to the dependency tree, even though the first statement should suffice:
3. Removing JAR files: When you delete the JAR file, Android Studio currently not does handle removal of these .JAR files in the build.gradle file. If you intend to remove one, you should check the build.gradle to see if you also need to delete the line that deals with importing the JAR file.
4. Adding library dependencies: If you're trying to add another library such as the Facebook SDK, which will usually reside outside the scope of your main app, you need to update the settings.gradle in the parent directory to reference this module. For instance, if you copy the contents into a Libraries/facebook directory, you need to have the 2nd extra include statement:
If you try to use the compile project statement without the changes to settings.gradle, you will likely see an error about "Project with path :Libraries:facebook could not be found in project :app". If you want a working example, check out Jonathan Azoff's repo: https://github.com/azoff/codepath-twitter-client
For more context, see the section on Multi project setup at http://tools.android.com/tech-docs/new-build-system/user-guide.
5. Use the Gradle wrapper option. When you select the option, you are effectively generating a few files that enable other people who check out your project not to have to install Gradle themselves once you've generated the files needed to bootstrap the process. It is the recommended that you check these files into your source code to allow others to be able to compile your program successfully.
The exact Gradle version that will be downloaded and used to build your app is defined in the gradle/wrapper/gradle-wrapper.properties file. The downloaded version is usually cached in ~/.gradle/wrapper/dists.
6. Unsure about whether Android Studio is creating build issues? If you'd like to see if the gradle build succeeds, you can always run things by executing Gradle at the command line. Within your project, you can type "bash gradlew assembleDebug" or "gradlew.bat assembleDebug", which will attempt to compile and build your Android app.
7. Seeing Duplicate files copied in APK errors? Double-check your build.gradle files in both your main Android app and your libraries. Your main app should be using the 'android' plug-in, while your libraries should be using the android-library plugin.
I made the mistake of trying to define all libraries with the 'android' plugin, triggering this issue possibly since Gradle didn't know how to resolve duplicate AndroidManifest.xml and other shared library files.
For instance, your main app should have the android plugin:
Your other libraries should be using the android-library plugin:
Regardless, here are a few things that I thought are useful to know if you're using Gradle and Android Studio:
1. There are multiple build.gradle files. When you create a new project, Android Studio creates two build.gradle files. The first is the parent that encompasses all modules in your project, while the 2nd is the one that pertains directly to the app. You will in most cases be dealing with this second build.gradle file. You do not really need to make any changes to the 1st one, which is blank by default.
2. Adding JAR files: When you drop a file into the designated libs/ directory, the changes are usually updated in the local build.gradle file. You can see the changes reflected in the dependencies section. For instance, if you drop the android-async-http-1.4.3.jar, it gets explicitly added to the dependency tree, even though the first statement should suffice:
dependencies { compile fileTree(dir: 'libs', include: ['*.jar']) compile files('libs/android-async-http-1.4.3.jar') }The addition of the 2nd is redundant because of the 1st statement -- however, if you add a library file outside the libs/ directory, presumably it will be automatically added to this list.
3. Removing JAR files: When you delete the JAR file, Android Studio currently not does handle removal of these .JAR files in the build.gradle file. If you intend to remove one, you should check the build.gradle to see if you also need to delete the line that deals with importing the JAR file.
4. Adding library dependencies: If you're trying to add another library such as the Facebook SDK, which will usually reside outside the scope of your main app, you need to update the settings.gradle in the parent directory to reference this module. For instance, if you copy the contents into a Libraries/facebook directory, you need to have the 2nd extra include statement:
include ':app' include ":Libraries:facebook"If your app intends to depend on this SDK, you also need to reference it inside the build.gradle of your app:
dependencies { compile project(':Libraries:facebook')
If you try to use the compile project statement without the changes to settings.gradle, you will likely see an error about "Project with path :Libraries:facebook could not be found in project :app". If you want a working example, check out Jonathan Azoff's repo: https://github.com/azoff/codepath-twitter-client
For more context, see the section on Multi project setup at http://tools.android.com/tech-docs/new-build-system/user-guide.
5. Use the Gradle wrapper option. When you select the option, you are effectively generating a few files that enable other people who check out your project not to have to install Gradle themselves once you've generated the files needed to bootstrap the process. It is the recommended that you check these files into your source code to allow others to be able to compile your program successfully.
gradlew gradlew.bat gradle/wrapper/ gradle-wrapper.jar gradle-wrapper.properties
The exact Gradle version that will be downloaded and used to build your app is defined in the gradle/wrapper/gradle-wrapper.properties file. The downloaded version is usually cached in ~/.gradle/wrapper/dists.
6. Unsure about whether Android Studio is creating build issues? If you'd like to see if the gradle build succeeds, you can always run things by executing Gradle at the command line. Within your project, you can type "bash gradlew assembleDebug" or "gradlew.bat assembleDebug", which will attempt to compile and build your Android app.
7. Seeing Duplicate files copied in APK errors? Double-check your build.gradle files in both your main Android app and your libraries. Your main app should be using the 'android' plug-in, while your libraries should be using the android-library plugin.
I made the mistake of trying to define all libraries with the 'android' plugin, triggering this issue possibly since Gradle didn't know how to resolve duplicate AndroidManifest.xml and other shared library files.
For instance, your main app should have the android plugin:
buildscript { repositories { mavenCentral() } dependencies { classpath 'com.android.tools.build:gradle:0.12.+' } } apply plugin: 'android'
Your other libraries should be using the android-library plugin:
buildscript { repositories { mavenCentral() } dependencies { classpath 'com.android.tools.build:gradle:0.12.+' } } apply plugin: 'android-library'You can also take a look at the explanation at http://tools.android.com/tech-docs/new-build-system/user-guide that covers the different between a project and a library project. A project will generate an Android Application Package (APK) file, while a library project will generate an Android ARchive (AAR) file. Since you can only have one APK file, it makes sense that you're only allowed to define one project. Android Studio currently doesn't prevent you from making this mistake however.
8. New versions of Gradle may require the Android Gradle plugin to be updated too.
When you add the "apply plugin: 'android'", you are in fact instructing Gradle to use the Android Gradle plugin, which Google maintains at https://android.googlesource.com/platform/tools/base/+/master/build-system/gradle/src/main/groovy/com/android/build/gradle/internal/). This plugin helps inform Gradle where to find your Android SDK, which is usually defined in local.properties.
The evolving nature of Gradle and the Android Gradle plugin means that the versions you need may need to be updated. You should always check the gradle/wrapper/gradle-wrapper.properties and the build.gradle file to verify that the versions you are attempting to use are compatible. Check the Android tools link http://tools.android.com/tech-docs/new-build-system to verify.
9. Seeing cryptic "failed to find Build Tools revision" errors?

Double-check the Android SDK Build Tools version specified in your buildToolsVersion reflects the one that's installed i Android SDK Manager. You may need to tweak your buildToolsVersion to be updated to one that you have installed. Similarly, you may also need to do the same for the compileSdkVersion definition.
10. Still confused? Read the writeup of new Android Gradle plugin build system.
When you add the "apply plugin: 'android'", you are in fact instructing Gradle to use the Android Gradle plugin, which Google maintains at https://android.googlesource.com/platform/tools/base/+/master/build-system/gradle/src/main/groovy/com/android/build/gradle/internal/). This plugin helps inform Gradle where to find your Android SDK, which is usually defined in local.properties.
The evolving nature of Gradle and the Android Gradle plugin means that the versions you need may need to be updated. You should always check the gradle/wrapper/gradle-wrapper.properties and the build.gradle file to verify that the versions you are attempting to use are compatible. Check the Android tools link http://tools.android.com/tech-docs/new-build-system to verify.
9. Seeing cryptic "failed to find Build Tools revision" errors?

Double-check the Android SDK Build Tools version specified in your buildToolsVersion reflects the one that's installed i Android SDK Manager. You may need to tweak your buildToolsVersion to be updated to one that you have installed. Similarly, you may also need to do the same for the compileSdkVersion definition.
android { buildToolsVersion "19.1.0"
10. Still confused? Read the writeup of new Android Gradle plugin build system.
Friday, July 18, 2014
Where Google keeps the Gradle source code..
Go to:
https://android.googlesource.com/platform/tools/base/
Instead of downloading the entire Android source code base, you can click on one of the branches and click on the 'tgz' link to download a snapshot of the tree.
https://android.googlesource.com/platform/tools/base/
Instead of downloading the entire Android source code base, you can click on one of the branches and click on the 'tgz' link to download a snapshot of the tree.
Sunday, July 13, 2014
How to read serial flash chips
Serial flash have become so prevalent in all electronic devices, replacing parallel based ones. Wondering how you can read off the data from them? The UM232H USB device can be used for this purpose. It can be purchased from Mouser.com or DigiKey.com for $20. In order to power the device, you need to wire things up based on Section 7.2 of the spec -- the VIO/3v3 and 5VO/USB pins should be shorted accordingly:
To verify the USB device is powered, an LED light should be lighted once you plug a Mini-USB cable to the connector. If you're using OS X, you can check that the device is connected on USB (system_profiler SPUSBDataType).
Product ID: 0x6014 Vendor ID: 0x0403 (Future Technology Devices International Limited) Version: 9.00 Serial Number: FTVRYH0J Speed: Up to 480 Mb/sec Manufacturer: FTDI Location ID: 0x14500000 / 10 Current Available (mA): 500 Current Required (mA): 90
In addition to the UM232H device, I ordered the Winbond W25Q serial flash chips from DigiKey, an SOIC to DIP adapter, and a small breadboard with wires to connect the chips. I originally ordered the Adesco AT45 SPI flash chips from Mouser, but these chips were too wide for the 150mil SOIC to DIP adapter I ordered. In order to make these chips work, I had to order the 208mil SOIC to DIP adapter, which is currently taking 2-3 weeks from China to arrive.
In order to discern what pins 13-16, I had to look at the circuit schematic, which shows that ADBUS0-ADBUS3 corresponds to the AD0-AD3 pins of the UM232H device. Note that the UM232H spec also has things listed according to the target device relative to the UM232H device. For instance, TDI/DO, means the wires should run to the DI input of the SPI Flash and match the data output of the UM232H device. I've included the schematics from the UM232H and the W25Q serial flash chip below:
Using the pyftdi project, I was able to issue the JEDEC command to get the manufacturer and device information back from the chip, which helped confirm that everything was working. The 0x403 and 0x6014 correspond to the vendor and product ID of the UM232H USB device:
from serialflash import SerialFlashManager from pyftdi.pyftdi.spi import SpiController flash = SerialFlashManager.get_flash_device(0x403, 0x6014) print flashI had to make a few tweaks to ensure that the JEDEC information could be correctly. The setup is currently using somewhat long wires between devices, so my goal is to reduce this length since it takes a couple of tries before the device returns the manufacturer ID correctly.
Subscribe to:
Posts (Atom)