Austin Summit Ironic Social

·

Since I was local, I decided to host an Ironic Social so everyone had a chance to get to know each other better outside of “work”. It proved to be harder than I thought to find a place close to the convention center with a wide selection of beers, wine, and mixed drinks, and a varied-enough menu to satisfy those with dietary restrictions, and large enough to handle 30-40 people at once. I talked to several places and decided on the Waller Creek Pub House. It was bar-service only which seemed to be alright, despite some lines at first when 30 people showed up and wanted to order a drink or some dinner. The staff did their best to accommodate us along with their normal Thursday night crowd. It was well attended with at least 30 showing up. I hope everyone had a good time, I certainly did.

IMG_0070 IMG_0080 IMG_0084 IMG_0086

Sharing Best Practices BoF: Third Party CI Working Group at Liberty Summit

·

We held our second Third Party CI Working Group meet up at the Vancouver OpenStack Liberty Summit. Because of limited space and a proposed user session, we were grouped together with the previously proposed Sharing Best Practices in Setting up and Operating OpenStack CI Loops Birds of a Feather session. Despite the fact that we have more that 100 CI systems, and that we had more than 100 attendees sign up, we had plenty of space. I was worried about the available space since the room only seated about 50 and we had 80-100 in Paris. I’m not exactly sure why we didn’t have the same turnout, maybe too short of an advertising runway after the cross-project sessions were finally decided, or the small space assigned in the back of the second building, regardless, it was really nice to meet more friendly CI operators and share experience. I estimate about 35-40 attended.

The Xen folks welcomed us with open arms into their BoF session. We decided to split the meeting into their CI overview and experience, and then to leave time for working group discussions. Stefano Stabellini and Bob Ball, both from Citrix, presented their experience bringing up a CI system for testing Xen + Libvirt, and explained the difference between XenServer driver testing and what they were doing for Xen + Libvirt.

IMG_20150520_140428796_a

Bob Ball presenting

Following this, we had some time to discuss some of the agenda items agreed upon in the weekly Third-party CI working group meetings. These items are listed in the etherpad for the session, so I won’t repeat them here. As always, time passed way too fast and we got little decided to further the work items identified in the etherpad.

Just like I did at the Paris session, I asked those attending a few questions and did a quick show of hands and came up with some somewhat surprising answers:

  1. The majority attending had already set up, or were in the process of setting up a third party CI system to test a driver or OpenStack on a different platform.
  2. There were only a few that were there to learn about OpenStack third party CI in advance of starting their test system.
  3. From those in #1 that had a system, only a few had even heard of the third party CI meetings we hold every week.
  4. Most did not monitor the openstack-dev maillist or ever get on IRC. Wow.
  5. Everyone I talked to said they would be willing to share tools they had written.

As a result, I did a quick overview of the meetings that are held weekly and urged everyone to get involved. My takeaway actions are:

  1. We must increase participation. The questions is, how do we reach teams if they do not monitor email or get on IRC?
  2.  We need to do a better job of advertising that help is available.

If you are reading this, please come to the weekly meetings that fit into your schedule. We can improve the consumability of OpenStack CI and our test systems by sharing our experience and tools. We have a repository now, and there are already some good tools there contributed from other systems. Spread the word.

Thanks everyone that attended the BoF session in Vancouver!

 

Third Party CI Tools Repo

·
The OpenStack third party CI working group has been discussing a way to share tools, configurations, and plugins best practices. This was an idea that started at the Paris summit because teams that are operating external CI systems have each created tools that make their life as CI operators easier. These tools are potentially valuable to all CI systems, and need to be shared. Now we have a way to share them.

I created this new repo with the following CI operators as initial cores in addition to myself. They were selected and voted in because they have consistently attended the working group meetings, helped fellow operators, and pushed to move work items forward in the community. They are: Ramy Asselin (asselin), Patrick East (patrickeast), Steve Weston (sweston), and Mikhail Medvedev (mmedvede).

Since this is a StackForge project, it shares the OpenStack patch process.  If you are interested in sharing something you have written that makes your CI life easier, please come to a working group meeting and discuss, or just submit a patch.

Clean up that swift object store

·

My team is using an OpenStack swift object storage service through SoftLayer as a way to publish the required logs to the community from our PowerKVM Continuous Integration testing. The test system ran like this for several months before we determined how to set the expiration on an object to have swift expire and remove a log after 30 days. No big deal really until we started to run out of space on our object store allocation and were being charged overages. This was not major expense, but it wasn’t right to leave all these old logs around from before we started expiring objects. They needed to be cleaned up.

I started out thinking that I would just go back and set the X-Delete-After header for the object, in this case a log file, and the swift-object-expirer daemon would eventually remove it when it tidied up. It seamed like the right thing to do, but did not work in practice. Try as I might, I could not programmatically set the X-Delete-After header for these old existing objects. Then I found this statement, “As expiring objects are added to the system, the object servers will record the expirations in a hidden .expiring_objects account for the swift-object-expirer to handle later”. I believe the key here is “As expiring objects are added”, meaning when we originally push the log file object to swift. Ah…so apparently a X-Delete-After header cannot be added to an existing object.

Well, fine. Then I’ll just create a tool that removes all the old objects. I decided that the way to do this was to look at the last_modified attribute of an object, compare this with a provided expire-older-than date, and remove anything older using the python swift client delete_object method. It just took a few iterations to get this working properly. I created a test container and filled it with a few random text files to use as an test target. Once I was happy with my new delete tool taking a supplied container and a delete-older-than date and having it remove all objects in my test container, I set it loose on the containers holding our PowerKVM CI logs. This was not without problems, as it would randomly fail after 20 minutes or so. I just caught the exceptions to continue, but then later did some debugging. A quick investigation indicated a possible scenario that when there are a lot of requests coming in, they can be queued. The client would then retry on the delete and fail, because the object was then marked for deletion from the queued request before the retry could complete. I have contemplated opening a defect on this, but looking at other swift client code, it seems to be a common practice to catch these HTTP errors and continue. All in all, no loss, it was a great learning experience for me.

Overall the process result was a success! My new del_objects.py tool cleaned out the old log objects and in the process removed more than 380 GB of old log files. I will be happy to share this and other swift tools that I put together, they aren’t much really, but might be good for an example to use for creating your own super utility. I must get approval from IBM to share them, and that is in progress. I will share them either in my personal github account or maybe on a new Third Party CI Working Group repository created for this, stay tuned.

Third-party CI work group at Kilo Summit

·

At the beginning of this year,  I was being asked to share my team’s experience standing up PowerKVM CI with teams that were just getting started.  Jay Pipes, Anita Kuno and the OpenStack infrastructure team were feeling overwhelmed by all the new third-party CI systems. So, after a brief discussion, we agreed to start this work group after the Atlanta Design Summit. We set out to help improve the third-party testing experience in OpenStack and have now grown into a large group of CI operators.

As a result, we held our very first meeting at the Paris Kilo Design Summit Third-Party CI work group session. The session event page showed 89 signed up and that number was close since I was estimating around 80-100 attending.  It was such a pleasure to get to meet everyone I have been interacting with for the past several months.

Third party CI discussion led by @anteaya and @kurtrtaylor

(Picture by Elizabeth K. Joseph)

The session started with a background overview by Anita. I then led the discussion on the agenda items agreed upon in the weekly Third-party CI meetings. These items are listed in the etherpad for the session, so I won’t repeat them here.

During the session, I asked those attending a few questions and did a quick show of hands and came up with some interesting answers:

  1.  The majority of CI operators attending had contributed at least one patch to OpenStack. I was somewhat surprised with this response. After discussing it and thinking it through, I realized that this made sense. The operators that understand how to participate in an open source community, understand how important it is to come to design summit.
  2.  Almost everyone attending the session knew about the Third-party CI weekly meetings. But, not nearly that many actually attend.
  3. Most Third-party CI systems have some functionality that they did not get from upstream CI, that they had to create for themselves.  Most of those system teams are willing to share what they have created and learned with the third-party community.

Time in the session went way too quickly. We spent a good portion discussing how to bootstrap teams, improve docs and finding alternate meeting times, but then had to rush through improving reliability and better self-regulating. I spent the rest of the week in numerous hall discussions on these topics.

My takeaways are:

  1.  Figure out how to help the operators that didn’t come to summit become contributors to OpenStack community, and to participate in the Third-party CI work group.
  2.  Get other third-party operators to start contributing to the care and feeding of this work group and the weekly meetings.
  3.  Create an additional meeting time for those teams that would like to attend the weekly meeting, but can’t because of time zone differences.
  4.  Help coordinate and participate in a work group focused on improving third-party CI documentation.
  5.  Pull together the efforts to contribute a third-party CI monitoring tool and dashboard (more on that in a later post).

We are now at a place where we as third-party CI systems can organize and help improve the consumability of OpenStack CI and ourselves in the process.  If you were not able to attend the summit session, I encourage you to get involved, attend a weekly meeting if you can, and be on the lookout for an additional meeting time that would be more convenient.

Thanks everyone that attended!

Consolidation

·

If you have followed the Linux Sound blog in the past and found your way here, welcome!

I still am very interested in embedded Linux and audio/music/sound support, but my current assignment to OpenStack keeps me very busy. Since I have not updated that blog in some time, and wanted to continue posting about an even broader set of topics, I have moved and merged that mission and my new interests into this one consolidated place.

Stay tuned for more entries on OpenStack, Linux, and of course, Linux Sound. Thanks!

A Second Android UCM

·

Now there are two. Yes, there are now two different UCM prototypes for Android on Pandaboard. But why?  Well, the first, tinyHAL, came out of conversations with Mark Brown and others, as described in my earlier post.  The second came from later discovering that Gabriel Beddingfield had implemented a similar PoC for Blaze. They are both good starts and both have their pluses, and now both run on Pandaboard with the Linaro Android builds.

The first, tinyHAL is available at here.  The second, android-ucm, is available here.  I have also facilitated the relicensing of the dependencies for android-ucm and deposited them here and here.  Please refer to the README in each of these repositories for the instructions on how to build these components in a Linaro Android build (edit: Linaro has removed these git repos, contact them directly for this source).

So, which one should be used and enhanced? Well, I like the XML mapping from android-ucm, and I like the XML configuration formats from tinyHAL. I suspect that the way forward should be to decide the best features, and then combine, refactor, and merge efforts. For now, try them both and see which one you prefer. As always, contributions are always welcome and feel free to contact me or linaro-dev@lists.linaro.org if you have comments.

UCM for Android

·

Following the ASoC Embedded ALSA Conference and the Linaro Multimedia Summit last year, I put plans in place to bring UCM to Linaro for both the Ubuntu-based builds and the Android builds.  We are seeing that happen.  UCM functionality is improving in Ubuntu and now is coming to Android.  Based on the initial drop of tinyhal by Mark Brown, I now have a prototype running on Pandaboard.  This code should be viewed as a proof-of-concept framework for UCM, not a complete fully-functioning implementation.  The code supports basic playback and will need to be enhanced for a full dynamic verb set.  I have created a git repository for my work here (edit: Linaro has removed this git repo, contact them directly for this source).

To give it a try, you will need to follow the Linaro Android instructions for building from source, which I recently blogged here.  Follow the README instructions for copying files to their needed locations.  You can test playback with tinyplay or the Android music player app.  The configuration file sets the initial volume, but the Android settings work normally for adjusting this level.

The remaining needed work will continue over the next several weeks, adding support for other mixing and routes.  I am also looking to unify the file formats between Ubuntu and Android.  Stay tuned for that proposal.  As always, contributions are welcome.

Setting up an Android Build

·

Yes, it has been too long since I blogged, I won’t bore you with the details of why.  Instead, I’ll kick this off again with a step-by-step post on how to set up a system to do your first Android build.

Linaro has a great new Android homepage with lots of useful links on it, go explore the info there http://wiki.linaro.org/Platform/Android

I did my installation on Ubuntu 11.10 Oneiric, so this post will make the most sense if you do too.  Put away that P3 box in the corner you were planning on using.  It seems to be a wise requirement to use a quick box with x86_64 support. You will also need ~30G of free space on your drive an a good amount of memory, plan for it. So, go do your AMD64 install of Ubuntu Oneiric desktop and then come back.

The AOSP “Initializing a Build Environment” page is your next stop. It is slightly out of date, but work through it  http://source.android.com/source/initializing.html  Under the “Installing required packages” section, there is a package change for Oneiric. The package lib32readline5-dev is not available, use lib32readline-gplv2-dev instead.

Next, go get repo.  To do this, follow the first 3 steps under the Quickstart section at http://wiki.linaro.org/Platform/Android/BuildSource  Wait on running the repo init command for now.

Install gcc-4.5. I used software center, and uninstalled 4.6 first. You may not have to do this as it may have caused me some extra fun for me. Regardless, when you have it installed, make sure that you have /usr/bin/cc and /usr/bin/gcc symlinked to gcc-4.5, not 4.6.

I also experienced a problem that was well documented here: http://unforgivendevelopment.com/2011/10/05/solved-cyanogenmod-build-issues-on-debian-wheezy-testing/  As this post suggests, running “sudo ln -s /usr/include/`uname -m`-linux-gnu/asm/ /usr/include/asm” fixed the problem.

Now it is time to decide on the code to build.  I was interested in reproducing the 12.01 Linaro Android “tracking” build because it had the patches needed to make audio work on the Pandaboard. The best way to do this is to follow the “How to Get and Build the Source” on the android-build.linaro.org page for the build you are interested in, in my case, https://android-build.linaro.org/builds/~linaro-android/tracking-panda-12.01-release/  If you follow this exactly (with any changes for your environment, of course), you shouldn’t have any problems. Also, I hope you aren’t in a hurry. The repo sync alone will take over an hour unless you have a super fast connection and the build will depend on your host, but mine takes a good long time.

When the build is complete, check the build log in the root directory to make sure there wasn’t any issues. To use the build you just made, refer to the instructions “How to Use Prebuilt Images” on the same Linaro Build page you used in the step above. The boot, system and userdata compressed tarballs are available in the ~/android/out/target/product/<board> directory. Substitute <board> with your dev platform, in my case that was “pandaboard”. Blast these onto a SD card just like you’d do with the prebuilt images and you’ll be up an running on your own Android build.

That’s all there is to it. Hope this helps, I tried to capture everything that I went through to get it working. Please let me know if you find any omissions or problems so that I can edit this post with any corrections.

Linaro Multimedia Mini Summit

·

Since the Linaro multimedia working group didn’t get to plan at the last Linaro Developers Summit, we held our own mini-summit at the beautiful IBM campus in Austin. It has been 3 weeks and I am just now able to think about other things like updating this blog due to the flurry of planning activities that followed. There are other factors at play too, one of them is I have agreed to take on the role of team lead for the multimedia working group. Even though my view now encompasses all multimedia, I will try to keep to this blogs original intent of writing about Linux sound and specifically ARM audio.

The Linaro multimedia working group,  often referred to as MMWG, needed to hold the summit to share our plans and discuss the requirements for the upcoming development cycle. It was very well attended by key technical folks from all areas of ARM-based multimedia. Attending were representatives from Linaro, IBM, NVidia, Freescale, Texas Instruments, Collabora, ST-Ericsson, ARM and others as well as several who were able to dial-in. I had organized meet-ups before, but it was the first time I had organized an event this size. All in all, it went very well and I think everyone enjoyed the experience of meeting to discuss the challenges ahead of us.