Improving automated OpenELEC build

31Aug12

I’ve written before about my OpenELEC build bot, and even detailed how to run one for yourself in the cloud.

After the VPS I’d been using died the other day I had to rebuild my environment, and thought I’d take the opportunity to try some things out.

The key change is using a loop device for creating the SD card images. This is perhaps what I should have done from the start, but the easy availability of extra hard disks on the VMs I was using made me take the easy road. Thanks to root9.net for helping me along the way here. Using the create_sdcard script from OpenELEC I’ve made a few tweaks:

create_loop_sd
#!/bin/sh

################################################################################
#      This file is part of OpenELEC - http://www.openelec.tv
#      Copyright (C) 2009-2012 Stephan Raue ([email protected])
#
#  This Program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2, or (at your option)
#  any later version.
#
#  This Program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with OpenELEC.tv; see the file COPYING.  If not, write to
#  the Free Software Foundation, 51 Franklin Street, Suite 500, Boston, MA 02110, USA.
#  http://www.gnu.org/copyleft/gpl.html
################################################################################

# usage:   sudo ./create_sdcard <drive>
# example: sudo ./create_sdcard /dev/sdb

if [ "$(id -u)" != "0" ]; then
  clear
  echo "#########################################################"
  echo "# please execute with 'sudo' or -DANGEROUS!!!- as root  #"
  echo "# example: sudo ./create_sdcard <drive>                 #"
  echo "#########################################################"
  exit 1
fi

if [ -z "$1" ]; then
  clear
  echo "#########################################################"
  echo "# please execute with your drive as option              #"
  echo "# example: sudo ./create_sdcard /dev/sdb                #"
  echo "# or:      sudo ./create_sdcard /dev/mmcblk0            #"
  echo "#########################################################"
  exit 1
fi

DISK="$1"
if [ "$DISK" = "/dev/loop0" ]; then
  PART1="${DISK}p1"
  PART2="${DISK}p2"
  IMGFILE="$2"
  losetup $DISK $IMGFILE
else
  PART1="${DISK}1"
  PART2="${DISK}2"
fi

clear
echo "#########################################################"
echo "#                                                       #"
echo "#             OpenELEC.tv USB Installer                 #"
echo "#                                                       #"
echo "#########################################################"
echo "#                                                       #"
echo "#     This will wipe any data off your chosen drive     #"
echo "# Please read the instructions and use very carefully.. #"
echo "#                                                       #"
echo "#########################################################"

# check for some required tools

  # this is needed to partion the drive
  which parted > /dev/null
  if [ "$?" = "1" ]; then
    clear
    echo "#########################################################"
    echo "#                                                       #"
    echo "# OpenELEC.tv missing tool - Installation will quit     #"
    echo "#                                                       #"
    echo "#      We can't find the required tool \"parted\"         #"
    echo "#      on your system.                                  #"
    echo "#      Please install it via your package manager.      #"
    echo "#                                                       #"
    echo "#########################################################"
    exit 1
  fi

  # this is needed to format the drive
  which mkfs.vfat > /dev/null
  if [ "$?" = "1" ]; then
    clear
    echo "#########################################################"
    echo "#                                                       #"
    echo "# OpenELEC.tv missing tool - Installation will quit     #"
    echo "#                                                       #"
    echo "#      We can't find the required tool \"mkfs.vfat\"       #"
    echo "#      on your system.                                  #"
    echo "#      Please install it via your package manager.      #"
    echo "#                                                       #"
    echo "#########################################################"
    exit 1
  fi

  # this is needed to format the drive
  which mkfs.ext4 > /dev/null
  if [ "$?" = "1" ]; then
    clear
    echo "#########################################################"
    echo "#                                                       #"
    echo "# OpenELEC.tv missing tool - Installation will quit     #"
    echo "#                                                       #"
    echo "#      We can't find the required tool \"mkfs.ext4\"       #"
    echo "#      on your system.                                  #"
    echo "#      Please install it via your package manager.      #"
    echo "#                                                       #"
    echo "#########################################################"
    exit 1
  fi

  # this is needed to tell the kernel for partition changes
  which partprobe > /dev/null
  if [ "$?" = "1" ]; then
    clear
    echo "#########################################################"
    echo "#                                                       #"
    echo "# OpenELEC.tv missing tool - Installation will quit     #"
    echo "#                                                       #"
    echo "#      We can't find the required tool \"partprobe\"       #"
    echo "#      on your system.                                  #"
    echo "#      Please install it via your package manager.      #"
    echo "#                                                       #"
    echo "#########################################################"
    exit 1
  fi

  # this is needed to tell the kernel for partition changes
  which md5sum > /dev/null
  if [ "$?" = "1" ]; then
    clear
    echo "#########################################################"
    echo "#                                                       #"
    echo "# OpenELEC.tv missing tool - Installation will quit     #"
    echo "#                                                       #"
    echo "#      We can't find the required tool \"md5sum\"         #"
    echo "#      on your system.                                  #"
    echo "#      Please install it via your package manager.      #"
    echo "#                                                       #"
    echo "#########################################################"
    exit 1
  fi

# check MD5 sums
  echo "checking MD5 sum..."

  md5sumFailed()
  {
    clear
    echo "#########################################################"
    echo "#                                                       #"
    echo "# OpenELEC.tv failed md5 check - Installation will quit #"
    echo "#                                                       #"
    echo "#      Your original download was probably corrupt.     #"
    echo "#   Please visit www.openelec.tv and get another copy   #"
    echo "#                                                       #"
    echo "#########################################################"
    exit 1
  }

  md5sum -c target/KERNEL.md5
  if [ "$?" = "1" ]; then
    md5sumFailed
  fi

  md5sum -c target/SYSTEM.md5
  if [ "$?" = "1" ]; then
    md5sumFailed
  fi

# (TODO) umount everything (if more than one partition)
  umount ${DISK}*

# remove all partitions from the drive
  echo "writing new disklabel on $DISK (removing all partitions)..."
  parted -s "$DISK" mklabel msdos

# create a single partition
  echo "creating partitions on $DISK..."
  parted -s "$DISK" unit cyl mkpart primary fat32 -- 0 16
  parted -s "$DISK" unit cyl mkpart primary ext2 -- 16 -2

# make partition active (bootable)
  echo "marking partition active..."
  parted -s "$DISK" set 1 boot on

# tell kernel we have a new partition table
  echo "telling kernel we have a new partition table..."
  partprobe "$DISK"

# create filesystem

  echo "creating filesystem on $PART1..."
  mkfs.vfat "$PART1" -I -n System

  echo "creating filesystem on $PART2..."
  mkfs.ext4 "$PART2" -L Storage

# remount loopback device
  if [ "$DISK" = "/dev/loop0" ]; then
    losetup -d $DISK
    losetup $DISK $IMGFILE -o 1048576 --sizelimit 131071488
    PART1=$DISK
  fi

# mount partition
  echo "mounting partition $PART1 ..."
  if [ -d /dev/shm ]; then
    rm -rf /dev/shm/openelec_install
    mkdir -p /dev/shm/openelec_install
    mount -t vfat "$PART1" /dev/shm/openelec_install
    MOUNTPOINT=/dev/shm/openelec_install
  else
    rm -rf /tmp/openelec_install
    mkdir -p /tmp/openelec_install
    mount "$PART1" /tmp/openelec_install
    MOUNTPOINT=/tmp/openelec_install
  fi

# create bootloader configuration
  echo "creating bootloader configuration..."

  echo "boot=/dev/mmcblk0p1 disk=/dev/mmcblk0p2 ssh quiet" > $MOUNTPOINT/cmdline.txt

# copy files
  echo "copying files to $MOUNTPOINT..."
  cp target/KERNEL $MOUNTPOINT/kernel.img
  cp target/SYSTEM $MOUNTPOINT
  cp 3rdparty/bootloader/* $MOUNTPOINT
  cp openelec.ico $MOUNTPOINT
  cp README.md $MOUNTPOINT

# sync disk
  echo "syncing disk..."
  sync

# unmount partition
  echo "unmounting partition $MOUNTPOINT ..."
  umount $MOUNTPOINT

# cleaning
  echo "cleaning tempdir..."
  rmdir $MOUNTPOINT

# unmount loopback device
  if [ "$DISK" = "/dev/loop0" ]; then
    losetup -d $DISK
  fi

echo "...installation finished"

The other big difference is I've split up the script into the looping part that checks for changes on git and makes builds, and the post build part that creates image files:

loopit.sh
while :
do
 # Inner loop for git pull
 while :
 do
 GIT=$(git pull)
 echo $GIT
 if [ "$GIT" = 'Already up-to-date.' ]
 then
 echo 'Waiting half an hour for another pull at Git'
 sleep 1800
 else
 echo 'Kicking off the build and post build processes'
 break
 fi
 done
 # Delete old build
 rm -rf build.OpenELEC-RPi.arm-devel
 # Make release
 PROJECT=RPi ARCH=arm make release
 # Run packaging script
 ./lendit.sh
# Loop back to start of script
done

This lets me run the image script standalone if I need to do that:

lendit.sh
# Set env vars for release package name
TARBALL=$(ls -t ~/OpenELEC.tv/target | head -1)
BUILD=$(echo $TARBALL | sed 's/.tar.bz2//')
RELEASE=$(echo $BUILD | sed 's/.*-r/r/')
IMGFILE=~/OpenELEC.tv/releases/$RELEASE.img
# Copy release build to web server
sudo cp ~/OpenELEC.tv/target/$TARBALL /var/www
cp ~/OpenELEC.tv/target/$TARBALL /mnt/box/OpenELEC
# Unpack release
cd ~/OpenELEC.tv/releases
tar -xvf ~/OpenELEC.tv/target/$TARBALL
# Wipe virtual SD
sudo dd if=/dev/zero of=$IMGFILE bs=1M count=910
# Move into release working directory
cd ~/OpenELEC.tv/releases/$BUILD
#Run script to create SD card
sudo ../create_loop_sd /dev/loop0 $IMGFILE
#Get into right directory
cd ..
# Compress release file
zip ./$RELEASE.img.zip ./$RELEASE.img
# Remove image file
sudo rm ./$RELEASE.img
# Copy zipped image to web server
sudo cp ./$RELEASE.img.zip /var/www
cp ./$RELEASE.img.zip /mnt/box/OpenELEC
# Go back to OpenELEC directory
cd ~/OpenELEC.tv
# Send an email
python gmail.py "OpenELEC Build Complete - $RELEASE" $BUILD


3 Responses to “Improving automated OpenELEC build”

  1. I love your work m8! Been using your build for awhile now and I am completely new to all of these OpenElec and RPi.
    Just bought MPEG key and it works flawless to play .vob files and MPEG files. One thing thou…. I bought also VC1 key to play WMV but it didn’t work. So I heard OpenElec need to update it and support this format, is that true?


Leave a reply to Chris Swan Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.