Skip to content

A quick way to automate AWS EC2 Snapshots

Earlier I posted a way to cleanup snapshots. I’ve had a few emails asking how I automated the creation. Here is a quick command line (Linux) to automate the creation of the snapshots. Make sure you have your environment setup and the latest version of the EC2 Tools API installed and in the PATH. You can place this command as a cron entry, or simply run it from the command line.

for a in $(ec2-describe-volumes |grep -Ev "ATTACHMENT" |cut -f2);do ec2-create-snapshot $a; done;

All this is doing is grabbing the volume ID’s from the ec2-describe-volumes output – removing the extraneous information, and executing the ec2-create-snapshot command on each volume. The good thing about this is that it doesn’t have to be modified as you add new volumes. Please see my earlier posting on snapshot cleanup to have a full solution from creation to cleanup.

Quick and easy.

2010 07 10 Alvin Kreitman Uncategorized Comments (0) Permalink

Automating EC2 EBS Snapshot Cleanup

I’ve recently taken on the task of building and now administering a cluster of Amazon EC2 instances. The EC2 command line tools provide all of the basic functionality you’ll need as it relates to creating new instances, EBS volumes, snapshots, and nearly everything you would ever need to do with all of the assets. The one missing piece was a script to clean-up snapshots. The way snapshots work is they accumulate in your account’s built-in S3 area, and you pay for that.

So the problem in a nutshell is I have 10 volumes, each of which is cron’ed to be snapshotted at various times of the day (depends on the specific volume as to how often it is backed up). With 10 volumes, my S3 storage costs can get out of hand quite quickly. So I needed to develop a set of scripts that would scan my snapshots – and remove the oldest ones – so I’m not paying for that storage. It is important to keep a couple of snapshots for each volume (at the very least) – and in some cases, I’d like to keep several snapshots. For example, one of my volumes is responsible for storing the main database for the CMS. This is backed up once every two hours. For that specific volume, I’d like to always have my choice of the last 10 snapshots to restore. If the database all of the sudden becomes corrupt, it may be necessary to restore earlier backups to see where and when the corruption started. Other volumes may only require the last 1 or 2 snapshots. So this script needed to be flexible – in that I could specify how many backups I’d like to keep for each volume.

Continue reading ›

2009 11 01 Alvin Kreitman systems-admin Comments (3) Permalink

Twitter Lists – A nice addition – But how do you filter out the junk?

I have to admit that I’m not usually one to be excited about the so-called innovation that comes out of many of these larger internet companies. In fact – very few of the so-called “enhancements” really impress me. But I’ll give props to Twitter. The road-map they’ve put together is exciting to me as a user first, and as an application developer as well.

Perhaps the most exciting thing is integration of the lists experience into their API. As a developer, I can see where this will allow a more realistic way of integrating relevant Twitter’ers into content site. I can see where domain experts will build lists of experts in their field who, if they have something meaningful to say, could provide for some very useful and relevant integration into content sites. Like Robert Scoble’s list for Most Influential in Tech.

Continue reading ›

2009 10 30 Alvin Kreitman web-apps Comments (1) Permalink

Smart LAMP Application Backups using Amazon’s S3

Repeat after me: I must backup my LAMP applications!! If you’re like me – you’ve hacked your way through several iterations of backup scripts over the years. I can’t even imagine how screwed I’d be if my server went down without a backup. I have thousands of hours worth of code and ultra important and sensitive data.

In the past, I put together scripts that had hard-coded commands to tar/ftp the applications and data to a remote server. As I added new applications – or as applications would mature (and spread into other areas of the server), I would simply modify the original scripts to include the new stuff. So, of course (because I’m lazy sometimes), I had on several occasions inadvertently neglected to include something in my backup scripts. Over time, my backup scripts became ugly, unmanageable, and over-complicated.

So I came up with a new strategy. Why not develop a set of scripts that:
Continue reading ›

2007 11 01 Alvin Kreitman tech-ramblings Comments (3) Permalink

I apparently look a little like Kate Blanchett!

Go figure.  I found this cool image analysis tool that attempts to see what celebrities you might resemble…

Continue reading ›

2007 09 18 Alvin Kreitman Uncategorized Comments (2) Permalink

Don’t you love College Football?

OK, so I’m making my first real contribution to the WordPress community. I’m a freak for college football; I have been for as long as I can remember. A few years ago, I developed a widget in Yahoo Widgets (formally Konfabulator). It’s called liveSchedule, and it allows users to display their favorite team’s schedule including results, record, and current ranking.

So I had this great idea that I should make a WordPress Widget that does the same thing. I’m releasing it to the world today! You can see a live example on the bottom left of this page. For more information, I’ve setup this special page where you can get additional information and download it for yourself!

I’d love to get your feedback and suggestions for the next version!

2007 09 09 Alvin Kreitman tech-ramblings Comments (4) Permalink

Maury, if you would!

Hi everyone. Welcome to techkismet.com. What’s it all about? Who am I? You’ll find these things out on the pages of this site. But here is a sneak-peak; I hope to make this site a combination of an online professional and personal diary and offer my own nonsense opinions about the world of technology, and how it continues to be my passion after 20 + years.

Why techkismet?

Why not? No really. Kismet translates to “fate”. Unlike many who choose their educational curriculum and career path with the help of guidance counselors or aptitude tests, or even the family’s influence – my path was to be decided for me. Yes, ladies and gentlemen; technology is now and has forever been my “fate”.

Continue reading ›

2007 09 02 Alvin Kreitman nearly-nonsense Comments (0) Permalink