Thursday, August 1, 2013

USB Bootable Media Toolbox

First, let me acknowledge that this is not ready for prime-time!  I have only tested it with 2GB, 4GB, 16GB & 32GB media, although theoretically, it should work for all bootable media 512MB and larger.  It still contains the CrOS kernel from June 2013, because I haven't had time to debug the issues with the updated kernel.  At this point, you will need access to another machine running Debian (or Ubuntu or their progeny) or a working ChrUbuntu install on your Chromebook.  You'll still need another machine to install a distro on the media anyway, so I gave up on trying to eliminate this requirement.

The archive contains a "ReadMe.txt" with detailed instructions.  In the end, you'll feel like you just did 16 cartwheels to move only a meter, but it works.  Be aware that everything I have tried to streamline or clarify the process has failed to produce bootable media, so substitutions or shortcuts of any kind are definitely not recommended.  Follow the instructions exactly, and you should end up with a piece of bootable media that you can install a Linux distro on.  Without further ado, here's the download:

Toolbox
MD5

Once you have bootable media, but before you install a distro on it (Step 10), you can create a backup of the disk with gdiskdump (included) and then restore it to this condition easily in the future.  Once a distro is running from USB, moving it to the HDD or SSD (say to to replace ChrUbuntu) is a relatively simple task.

69 comments:

  1. How do you run ./fix-gpt.sh as root? I tried doing sudo, but it keeps telling me "Permission Denied". I then ran the commands in the bash script one by one, but it won't boot. Tried two different flash drives. Do I just have the worst flash drives in the world?

    ReplyDelete
    Replies
    1. I just tried it on 2 more flash drives, with same results. Am I just booting wrong or something?

      Delete
  2. I doubt the media is the problem, as I haven't yet found a drive that wouldn't boot. You don't really need to be root, since the script uses sudo.

    Are you attempting this from the chronos account in CrOS? From the Ctrl-Alt-F2 VT? If so, did you create a password for it? That seems to cause problems for some reason.

    ReplyDelete
    Replies
    1. I did everything until step 5 on my Linux Mint laptop. For step 5, I tried on my Chromebook and that's where I got the Permission denied issue. I just ran it in the crosh shell and tried doing "sudo bash" but got the same thing.

      Delete
    2. Boot ChromeOS but don't log in. Hit Ctrl-Alt-F2 and login as chronos, no password. Then cd to wherever you extracted the archive and try

      sh fix-gpt.sh

      As long as the chronos account has no password, this should work. If not, try

      sudo su

      and see if you get the colorful root prompt. Also, do you have ChrUbuntu installed?

      Delete
    3. The media should be bootable prior to step #5, but will only have 256MB of space.

      Delete
    4. I do have chrubuntu installed. Does that make a difference? Do I need to wipe it and try from stock ChromeOS?

      Delete
    5. No, not at all. But it gives us the option of running the CrOS utilities (like cgpt) from ChrUbuntu rather than CrOS. Generally speaking, it's easier to deal with issues from a complete Linux that also has the CrOS utilities. I'm pretty sure that ChrUbuntu included them by default, but ...

      Delete
  3. Running 'sh fix-gpt.sh' seemed to fix the "Permission Denied" error, but still won't boot. You think it'll boot before I try fixing the gpt tables?

    ReplyDelete
    Replies
    1. Just tried and it does for me. You are using Ctrl-U rather than Ctrl-D, right?

      You're obviously in Developer Mode with OS verification turned off. Are you running the developer firmware?

      Delete
    2. Sorry, my bad. I just assumed that everyone knew how to enable booting from USB. Review the requirements here:

      http://chromebooklinux.blogspot.com/2013/03/installation-method.html

      The links will take you to the official instructions.

      Delete
  4. Yeah, I'm hitting Ctrl-U and it'll just show a black screen for a second, then beep and kick me back to the OS verification screen.

    How do I check to make sure I have the dev firmware on? I was thinking it got installed with ChrUntu?

    When I run 'sudo crossystem' it says "dev_boot_usb = 1", so I'm assuming that booting from USB mode is enabled.

    ReplyDelete
  5. I think you're right about the firmware - necessary to boot anything except CrOS. Everything else sounds right, I'm just running out of ideas. Remember that every time it fails to boot, priority & tries will reset to zero and you need to 'sh fix-boot.sh' to try again.

    ReplyDelete
    Replies
    1. OK, rebooted my brain. So, have you EVER successfully booted anything from USB on your C7?

      Delete
    2. I can't remember. I compiled and installed ChromiumOS once, but I can't remember if it had to boot off of USB, or if I had to install it by running the recovery. It's been too long.

      Delete
    3. OK, then at this point, I would suggest that you download & try Hexxeh's Chromium OS for USB:

      http://chromeos.hexxeh.net/download/vanilla?&version=4028.0.2013_04_20_1810-r706c4144&type=usb

      It boots unaltered on my C7. Instructions are on his page:

      http://chromeos.hexxeh.net/

      This will provide us with an independant test.

      Delete
    4. I'll give it a try this weekend. Thanks for being patient with me!

      Delete
    5. So I dd'd Hexxeh's ChromiumOS onto a flash drive, stuck it in my Chromebook, turned it on and hit Ctrl-U at the OS verification screen and it did the same thing as I was getting before: black screen for a second and then just a beep...

      Delete
    6. This suggests to me that your C7's problem booting from USB is not the image. Most likely a firmware setting, the media itself or even hardware. If you'll post the complete output from 'sudo crossystem' I'll compare your settings to mine. That seems like the next logical step.

      Delete
    7. I ended up wiping my Chromebook and starting from scratch and got it to start booting from USB. My problem now is getting Crunchbang to boot from USB.

      I couldn't get it, so I figured I'd just try copying everything from the /dev/sdb7 partition to the /dev/sda7 partition, and I was able to get it to boot, but for it has errors booting and mounts / as read only so I can't do anything inside #!, but it does boot. So I'll keep playing with it and we'll see how it goes.

      Delete
    8. I wanted to suggest that but didn't want to overwhelm you. I'm glad you got it to boot! I had no issues with #! booting from USB that I can recall. You might want to re-format partition #7 and install a fresh copy.

      I really can't comment on the issues with the HDD since I've never moved a rootfs that way and you're now running on Jay Lee's kernel which I know nothing about.

      Delete
    9. This comment has been removed by the author.

      Delete
    10. EDIT2
      This may be a bit to late to help M0ngr31 but in case anyone else hits this problem, here's what I found

      === Enable developer mode ===
      Invoke Recovery mode, you hold down the following keys:
      ESC, F3/Refresh, Touch the Power button
      This will display the prompt "Chrome OS is missing or Damaged Please insert a recovery USB stick."

      Press ctrl, and D.
      This will display the prompt "To turn OS verification OFF, press ENTER. Your system will reboot and local data will be cleared. To go, back press ESC."

      If you are happy to proceed then press Enter

      Wait and the system will beep twice. Then the system will go into developer mode which can take 5 minutes. After a reboot you will get the prompt "OS Verification is OFF Press Space to re-enable" followed by two beeps

      You will be brought back into a new Chrome Install, where you normally select, your language, keyboard, and network.

      Press and hold the following keys to get a shell:
      ctrl, alt, F2/Forward (->)

      type root to login

      type the following to enable booting from a USB, booting from the SD will not work
      crossystem dev_boot_usb=1
      crossystem dev_boot_legacy=1
      crossystem dev_boot_signed_only=0

      type reboot to reboot the system
      At the "OS verification is OFF" prompt press ctrl, and then at the same time the letter U to boot from the USB Stick.

      Delete
    11. Thanks for documenting that process, Ken. I have tended to assume that the user did all of that within hours of taking it OOTB!

      However, I think I've always worked from the 'chronos' account. I generally avoid 'root' if possible.

      Delete
  6. Worked perfect the first time!

    I now have a Minty fresh install of Linux Mint 15 Olivia on the hard drive.

    Thanks for a great tool box

    ReplyDelete
    Replies
    1. I am so relieved to hear that! Thank you for the report!

      Delete
  7. Hi

    I have installed linux mint onto a usb and is booting succesfully , the only problem i have now is how do i install linux mint onto the acer c7's hard drive.You said it is relativly easy but when i tried to movie the os to the hard drive it says i dont have permissions to move a script.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. I'm sure I did this the hard way, but here is what I did.
      I built the USB stick following the readme file.
      Once the Mint install completed on the USB stick,
      copied all the Mint files (sdb7) to a sdcard.
      Booted on the USB stick.
      Mounted and deleted all the files on SDA7 partition.
      Mounted and copied the fresh files from the sdcard to sda7.

      Hope this helps.

      Steve

      Delete
    3. So you just replaced the ChrUbuntu partition with LM?

      Delete
    4. I did not do it as a partition, I did it by deleting all the chrubuntu files and then copying the Mint files to sda7.

      Delete
    5. Fascinating! It would never have even occurred to me to try moving files. I learned long ago that most OSes don't like that.

      Keep in mind that you're still running the kernel installed by ChrUbuntu on partition #6. Not a problem for you apparently, but in some cases that could be a 64 bit kernel.

      I'm also intrigued that you got LM to boot as I was only successful with LMDE. Which flavor (Desktop Environment) did you use?

      Delete
    6. I'm running Cinnamon.
      Yes, the kernel is on sda6. I tried it since I was going to have re-size the partition if I did it the other way. I wanted to have a fresh clean boot the first time it booted off the hard drive.

      Delete
    7. Hi Steve

      How I have managed to delete partition #7 on the acer c7's hard drive but I am unable to copy the system files from the usb to the internal hard drive.It shows up as somthing along the lines of "Cannot copy files , unable to read you do not have permissions to move a script"

      sorry if I am being noobish or somthing else

      Delete
  8. Interesting, I never tried Cinnamon since it was the heavier option. LM 15 Mate is on my list, though. Thanks!

    Yeah, your method allows you to retain the partition size you gave to ChrUbuntu with no further action required and re-partitioning the HDD is a risky proposition. It also may open the door for alternate install methods. Kudos!

    ReplyDelete
  9. Okay, I have Zorrin Os installed on my usb drive, and it boots, how do i go about actually installing it on my hard drive? I have chrubuntu already on it. I have an Acer C7

    ReplyDelete
  10. Boot into chrubuntu
    Copy all the files from sdb7 to a SD card.
    Boot on the USB drive
    Delete all the files on sda7 (rm -rf *)
    copy all the files from the sd card to sda7.

    ReplyDelete
    Replies
    1. Okay, but i thought that chrubuntu had a 64 bit kernel, and i am using a 32 bit install of Zorin. Would that be a problem?

      Delete
    2. It will be a problem if your ChrUbuntu is 64 bit. When the C7 was introduced, ChrUbuntu on every platform was 64 bit. However, for the last few months, it's been 32 bit on the C7. Open a terminal in ChrUbuntu and enter 'uname -m' to be sure. If the output is 'i686' then you're running a 32 bit kernel.

      Delete
    3. It outputs x86_64, could I just follow the same instructions, except use a 64 bit Zorin Os?

      Delete
    4. That might work, but I can't recommend it. In my experience, OSes are best handled as partitions & filesystems, not files. In any case, it would be simpler to merely replace the kernel on the HDD.

      Delete
    5. This is a nooby question, but how would I get the kernel off the usb stick, on the HDD?

      Delete
    6. Simply copy the partition as an object, then give it a new UUID, like this:

      sudo dd if=/dev/sdb6 of=/dev/sda6
      sudo cgpt add -i 6 -u $(uuidgen) /dev/sda
      sudo cgpt show /dev/sda

      Be sure to double check the P,T and S attributes before you reboot.

      Delete
  11. Okay thanks Steve I will try that now

    ReplyDelete
  12. I agree with Barry.
    I knew the install of chrubuntu I had on the hard drive was 32 bit. As Barry says, the kernel is on partition 6. When I built the USB stick I made sure to install a 32 bit version of Mint.

    The important piece of information from the method I used was the fact that I copied all the files from the fresh install on the USB drive to a SD card before I tried to boot the USB stick.

    When I did the copy to the hard drive I was booted on the USB stick. I made sure that I emptied partition 7 on the hard drive. I then copied from the SD card to sda7.

    Make sure when you copy files, that the source and destination files are not in use.

    ReplyDelete
    Replies
    1. Thanks for clarifying that critical factor, Steve, as I'm not sure everyone picked up on that.

      Delete
  13. Here is a Reddit thread about getting Crunchbang working. I was able to do this tonight and it is working great.

    http://www.reddit.com/r/chrubuntu/comments/1jq2ur/today_is_the_day_i_got_crunchbang_running_on_my/

    ReplyDelete
  14. Awesome!! Yeah, #! works very well on the C7, it's just a little spartan for my taste. But, if you value speed & efficiency over aesthetics, it's probably pretty close to perfect. Thanks for sharing the thread too!

    ReplyDelete
  15. Help for a newbie. Could you please provide details for step #10? I've successfully completed 1 - 9 but don't know how to install Mint on the USB drive. I've downloaded the ISO file and tried to install it using:

    dd if=linuxmint-15-cinnamon-dvd-32bit.iso of=/dev/sdb7 bs=1M

    But this did not work. Any help would be appreciated.

    Thanks.

    ReplyDelete
    Replies
    1. OK, dding the iso image to removable media essentially creates a pseudo-CD/DVD that will boot on most devices, i.e.

      dd if=LinuxDistro.iso of=/dev/sdb

      and can therefore be used just like a CD/DVD to install Linux somewhere else. Unfortunately, this same media is unusable on a Chromebook. You will have to burn the iso to CD/DVD or dd it to removable media (as shown above) then boot that on another machine to install Mint on partition #7 of your Chromebook-compatable bootable media. I prefer using optical media, if possible, because it's less confusing in the end. I hope that helps.

      Delete
  16. Hi Barry,

    I just discovered your blog, and I'm planning to try installing #! on my c710 with the USB method you've made available.

    I've used the installation guide from chromebook-linux.com and though I did get Debian 6/7 working, things like the touchpad, ethernet, etc weren't usable, even after installing multiple alternate drivers.

    In another post you mention configuring the touchpad in Debian, but I haven't seen anything about drivers, does it work out-of-box for you / your method?

    I hope to be back shortly with the install results.

    Thanks,
    -Chris

    ReplyDelete
    Replies
    1. Short answer: Yes! Everything should work OOTB.

      Long answer: Since we're using the Chrome OS kernel, we "transplant" the CrOS kernel modules (drivers) and CrOS utilities into our chosen distro's root filesystem. This is step 11 in the instructions for the USB Bootable Media Toolbox.

      Delete
    2. Awesome, I guess I've been doing the kernel transplant wrong, or using the wrong kernel.

      Still working on #! :) Here's hoping it works.

      Delete
    3. It should, I don't recall having any issues with #! Linux.

      A lot of Linux on Chromebook was 64-bit when the C7 was launched. But Chrome OS on the C7 has always been 32-bit only, so using 64-bit meant using a custom compiled Chromium OS kernel and non-OEM modules. My preference was to go to 32-bit Linux so we could utilize the stock kernel & modules installed by Google. In addition, most Chromebooks don't have enough RAM (2-4GB) to need 64-bit Linux anyway.

      Delete
    4. It's been a pretty long weekend, but I think there's finally a working copy of Debian Wheezy installed on my Chromebook! (gave up on #! :| )

      I've tried wrapping my head around GPT before, and couldn't quite get it, and have yet to see any really good documentation on most of the Chromebook utilities like cgpt and vbutil_kernel. That is to say I'm pretty lost with how the partitioning/booting on chromebooks operate, which has made this installation the single most frustrating thing I think I've ever done to get Linux on a device.

      I do know, though, that a 16G ssd is pushing it for me on the low space realm, and I really didn't care for the mess of partitions that Cromeos sets up by default. I think I've got everything running on just two partitions: KERN-A and ROOT-A with Debian on it.

      I don't think I've done anything of note, but I'll still list it if it can help someone, since it should work without needing any USB disks:
      -Modify the Chrubuntu script and turn it into a smaller script that just rips the firmware and modules directories (maybe cgpt too) and builds a new kernel.bin that points to /dev/sda2 as root.
      -Tar all of the files found by the above script, upload the tarball (tarball? that's what it's called right?).
      -Go to a desktop running Ubuntu/Debian. Download tar file. Install cgpt and vbutils (if you have x64 then I think there are prebuilt executables at chromebook-linux.com somewhere).
      http://packages.ubuntu.com/raring/cgpt
      -Create a .img file that's a few gigs large. Format it to ext4 or whatever. Use KVMs (virt-manager) to install Debian 7 to the .img file. Use expert install in Debian and skip installing GRUB. Finish installation and shutdown.
      -Mount the .img file, copy over the .tar and extract it. Push the files to the right folders. Edit the /etc/fstab to display a /dev/sda2 partition. Umount.
      -Shutdown the desktop and install the Chromebook's ssd in your computer. Reboot.
      -Use dd to zero out the ssd, then cgpt to format it with a KERN-A and ROOT-A. use dd to copy over the kernel. formatt the ROOT-A (dev/sdb2) as ext4 or whatever.
      -Mount both the img and ROOT-A (dev/sdb2). Copy over the files from img to ssd. Umount both. Shutdown desktop again and remove ssd, install back into chromebook.
      -Cross fingers and boot up the chromebook? (Took nearly a dozen iterations of this to get it right, i.e. all weekend).

      Anyway, I want to thank you for all the work you've made available, I seriously enjoy having a lightweight, fully-functional laptop!

      Delete
    5. Excellent! Thank you for documenting your learning process, I have no doubt that others will benefit. I certainly understand the frustration factor! If you attempt such a feat again, please let me know about your goal, as I can probably save you some work.

      Yes, the firmware expects at least two partitions: a kernel and rootfs with consecutive numbers (1 & 2 work just as well, at least from USB). The downside to abandoning Chrome OS is that you lose access to Google's automatic kernel updates, which have been frequent lately. Of course, it appears to me that the updates break things more often than not, but that's another discussion entirely.

      I agree that the 16GB SSD is restrictive for a full-blown current Linux distro. In such a case, retaining CrOS may not be an option.

      However, I personally would discourage anyone from voiding their warranty (again, another discussion) to avoid the purchase of a $10 flash drive. IMHO, a safer approach is to work with an OS on USB first and move it to the HDD/SSD only after you're certain that everything is working as it should.

      But, I do not mean to disrespect your contribution here! Everything that advances our collective knowledge about what's possible with a Chromebook is invaluable. Thank you so much and I'm glad that you're happy with the outcome! The C7 IS a pretty remarkable piece of tech.

      Delete
    6. The more you know... xP

      Next time I'll have to cool the jets and actually learn what to do first, the trial and error method definitely didn't pan out well.

      Also, I didn't think of that, but I had already voided the warranty and plan to upgrade the ssd if possible in the future, so I felt comfortable playing around with it. Otherwise the USB method is definitely the best plan!

      Kernel updates... I guess it'll wait until I get that new ssd and repeat the process. For right now though, Debian runs absolutely beautifully on the C7, I really couldn't be happier with the result.

      The shortened Chrubuntu scripts are going up on a local wiki while I try to figure out exactly how they work. Again, when the next install happens I'll finish it up and post back to get your review.

      Thanks again,
      -BBX

      Delete
    7. You are certainly more than welcome! I've been ignoring kernel updates myself for the last couple of months. But, there seems to be some talk among the Google developers of moving the C7 to 64-bit ChromeOS at some point in the future. I can't really see the advantage with the stock 2 or 4GB of RAM, but a lot of people have upgraded.

      Yeah, Wheezy (at least with LXDE) ran smooth as silk. I was really impressed with it.

      The trial & error method works (I know). It's just that the way Chromebooks were designed, there seems to be a lot of error required! But, hey, you're pleased with the end result - that's all that really matters!

      Delete
    8. This comment has been removed by the author.

      Delete
  17. I went through the Readme.txt and installed Fedora 17. After much trial and error, I made it through the entire process. I booted my usb drive just to get the error message: "Kernel Panic - not syncing: No init found. Try passing init= option to the kernel." Any idea on what went wrong?

    ReplyDelete
  18. Not really. That's an error message I associate with an incompatible distro, but Fedora 17 worked for me. Is it possible that the kernel can't find the rootfs? Make sure that your kernel & rootfs are on partitions 6 & 7. I assume the media was booting OK prior to the Fedora install?

    ReplyDelete
  19. The media was working fine prior to the Fedora 17 install. I did the install to partition 7. I did not do anything with partition 6. What should I have done with partition 6?

    ReplyDelete
  20. Nothing at all, it was already set up for you - evidenced by booting Ubuntu Core 13.04.

    I'd suggest re-installing but switch distros to avoid compounding the frustration. Fedora has been problematic. If you're fond of Fedora, it's certainly worth another attempt at some point, but I wouldn't want to see another failure now.

    ReplyDelete
  21. This comment has been removed by the author.

    ReplyDelete
  22. It looks like the bootable media archive you include with the download runs kernel 3.4. Do you know what is all involved with updating it to something newer?

    ReplyDelete
  23. That's correct. Well, I know some people have compiled newer kernels from the Chromium source and reported that they work. But, I chose to stick with the x86 (32-bit) kernel Google was using with Chrome OS on the C710 and later froze the kernel in the toolbox when an update broke wireless (I think) in a pre-existing install of Ubuntu 12.04(.2?). So, no, not really. In case you're not aware, stock firmware will only boot Chrome(ium) kernels in my experience, despite claims to the contrary.

    ReplyDelete
  24. So, how exactly would I go about copying a perfectly working USB install of, say, Elementary OS, to the HDD of my Acer C7 running a Chrubuntu-script-installed Xubuntu? I've tried this several times over using various methods I've found on /r/Chrubuntu, but it'd either fail at boot, throwing errors about mounting the file system, or would copy over but would break WiFi and Bluetooth (or in other words, all the things involving the Chromebook-specific drivers and modules).

    ReplyDelete
  25. Sorry about the delay - my internet is down more than it's up these days. I'm not exactly sure what's happening to you. Are you copying both the kernel partition & rootfs partition from USB to HDD? If it works from USB, then it should work from HDD as well. You'll need to chage the GUIDs of the cloned partitions. And you'll probably want to resize the rootfs partition after the fact, but that shouldn't interfere with booting it up.

    ReplyDelete
  26. Hi. So I was following your tutorial (which by the way is awesome!) and I am getting a bit stuck. I am not very versed in terminal command which might be the problem but I am unsure how to extract to the root fs on the usb. I have tried extraction to that pointed mounted, unmounted, not to mention just copying and other methods. Any help would be awesome. Thank you!

    ReplyDelete