Technical Notes

My online notepad

  • Social


  • Support

    Donate towards my web hosting bill!

Archive for December, 2013

Notes on creation of Windows AMI in EC2

Posted by Anton Khitrenovich on December 24, 2013

AWSCreating AMI from Windows instance in AWS is simple and easy. The whole process is well-documented, yet here are several notes that I made while experimenting.

Running Sysprep Tool
In the VMware world, you can convert a VM to a template at any moment. Deployment of multiple clones is made possible via Customization Specifications mechanism, where you define how to customize the new VM after cloning. As part of the customization process, vCenter will run Sysprep tool – after the instance was already cloned. In contrary, AWS wants you to run sysprep before the cloning, as preparation to AMI creation. You can do that via EC2ConfigService settings (see below) or manually, but former seems to be the preferred and recommended way.
The unwanted side effect of running sysprep on the original instance is that many of the original instance settings (instance name, Administrator account etc.) are actually reset during the cloning process. There are workarounds that can help you to deal with some consequences, but not all of them.

EC2ConfigService
The EC2ConfigService is Windows service provided and installed by AWS, which takes care about a lot of useful customizations behind the scenes. Examples are generation of Administrator password (one that you retrieve via “Connect” button in EC2 Management Console), setting of computer name, display of system information on the desktop wallpaper and a lot more. But the most important task is to deal with instance cloning – that is, make sure that stuff that is supposed to be unique is really unique and make this cloning process simple and easy.
Note that you cannot install this service on a Windows instance imported from other systems (for example, VMware vCenter). You’ll have to start your original instance from AWS AMI if you want to use EC2ConfigService there.

Instance Name
The name of the original Windows instance will be reset during sysprep process to the default semi-random name provided by AWS (in the form of “WIN-<something>”). The only workaround for that is to use “Set Computer Name” from EC2ConfigService settings. This flag instructs EC2ConfigService to (re)set system name according to the internal IP adrress – which remains constant during the instance lifecycle. The resulting name will not become much more readable, but at least it will not change during the AMI creation process. You’ll have to remove this flag manually on the cloned instances to set more meaningful names there.

Administrative Accounts
The Administrator account from the original instance will not make his way to the cloned instances, so any customizations made there will be lost. The workaround is to create an additional account with administrative permissions and perform the desired customizations there. There are many settings that will go this way too (for example, regional settings or pinned taskbar items), but some will survive – desktop shortcuts, windows explorer options and more.
Another tip is to choose “Keep Existing” in the “Administrator Password” section of EC2ConfigService configuration. While it won’t help you to retain current password in the new instances (because Sysprep tool on Windows 2008 resets it anyway), at least the Administrator password of your original instance will not be changed.

AMI Creation Steps
This is not an ultimate guide on how to create AMI, but rather a note/reminder to myself…

  1. Create a new EC2 instance from AWS AMI
  2. Create new administrative account and start using it instead of the build-in one
  3. (optional) Use “Set Computer Name” option from EC2ConfigService settings
  4. Customize your instance – install software, add desktop shortcuts etc.
  5. Start EC2ConfigService settings utility, go to the “Image” tab and perform the following:
    • (optional) Under “Administrator Password” section, select “Keep Existing” option
    • Click “Shutdown with Sysprep” button and confirm the prompt
  6. Wait while the Sysprep process completed and the instance shuts down
  7. Right-click the instance in the EC2 Management Console, choose “Create Image” and follow the instructions

Tip: Disk Size
When you launch an EC2 instance that you plan to convert to AMI later, pay attention to the disk size. Remember that the disk of the AMI must be at least that big, and the same applies to any instance that will be launched from the new AMI. Provisioning 10GB of spare storage in your AMI will cost you $1 per instance per month, which may be kinda painful in the yearly bill (do the math yourself).

Posted in HOWTO, Workarounds | Tagged: , , , , , | 2 Comments »

Properties resolution in Maven and its implications on Antrun plugin

Posted by Anton Khitrenovich on December 18, 2013

Two days ago I did some internal research in order to understand how do resolved properties propagate from Maven to Ant via maven-antrun-plugin. On my way I had to recall several rarely remembered facts and learn some new ones. The implications of their combination leads to pretty interesting results that are worth writing them down. At least, now I’ll know where to look for this information next time.

Fact #1
There are multiple sources for properties resolution in Maven – implicit project properties (like project.version), environment variables, user-defined properties (ones you use in properties tag or your pom.xml), Java system properties… The full list with detailed explanations is available in The Complete Reference online book.
Pay attention – when you “override” some user-defined property via command line parameter (-D), in fact you are setting a new Java system property. This works because system properties have higher precedence during properties resolution process.

Fact #2
All properties that are explicitly used in pom.xml are resolved before Maven lifecycle actually starts, even before the initialize phase. It happens when so-called “effective pom” (full project definition, including all implicit data and configuration settings propagated from parent project) is generated. BTW, you can see this in action by call mvn help:effective-pom on your project.

Implication #1
Modifications of project properties that happen during project lifecycle have no effect on the effective pom – it is just too late. Examples of such modifications include groovy scripts (via gmaven-plugin) and properties loaded from external files via maven-properties-plugin. So, why do we need them at all? Since they can be used by other plugins in runtime, when they are read directly from properties collection during plugin invocation.

Read the rest of this entry »

Posted in HOWTO | Tagged: , | 1 Comment »

How to attach Maven artifact from external build.xml file

Posted by Anton Khitrenovich on December 16, 2013

As you probably know, maven-antrun-plugin provides a custom AttachArtifact task that is able to attach artifacts generated by Ant to the current Maven project. The usage is really straightforward with inline Ant targets, but an attempt to use this task with external build.xml results in an error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (...) on project ...: 
  An Ant BuildException has occured: The following error occurred while executing this line:
[ERROR] C:\...\build.xml:158: Maven project reference not found: maven.project

[ The solution described here does not really work. See update at the bottom. ]

The solution is simple – external Ant invocation should inherit references from the inline target:

<plugin>
	<artifactId>maven-antrun-plugin</artifactId>
	<executions>
		<execution>
			<id>...</id>
			<phase>compile</phase>
			<goals>
				<goal>run</goal>
			</goals>
			<configuration>
				<target>
					<ant antfile="build.xml" target="..." inheritRefs="true" />
				</target>
			</configuration>
		</execution>
	</executions>
</plugin>

Update: It appears that this change eliminates the error, but the artifact is not actually attached. I’ve opened a bug report to the Maven Antrun Plugin team about that.

Posted in HOWTO | Tagged: , | Leave a Comment »

How to unpack RAR files on AWS-hosted Ubuntu servers

Posted by Anton Khitrenovich on December 10, 2013

It appears that regular unrar package is not available by default via apt-get on AWS Ubuntu servers. If you still need to unpack RAR files and prefer to install packages from default APT repositories only, the right package to install is unrar-free. It will do the job for you in most of the cases.

ubuntu@ip-10-0-0-38:/work/$ sudo apt-get install unrar-free
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
pike7.8 pike7.6 pike
The following NEW packages will be installed:
unrar-free
0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded.
Need to get 24.8 kB of archives.
After this operation, 101 kB of additional disk space will be used.
Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ saucy/universe unrar-free amd64 1:0.0.1+cvs20071127-2 [24.8 kB]
Fetched 24.8 kB in 4s (6,201 B/s)
Selecting previously unselected package unrar-free.
(Reading database ... 67423 files and directories currently installed.)
Unpacking unrar-free (from .../unrar-free_1%3a0.0.1+cvs20071127-2_amd64.deb) ...
Processing triggers for man-db ...
Setting up unrar-free (1:0.0.1+cvs20071127-2) ...
update-alternatives: using /usr/bin/unrar-free to provide /usr/bin/unrar (unrar) in auto mode
ubuntu@ip-10-0-0-38:/work$ 

Unfortunately, sometimes unrar-free is not enough – for example, when you have to unpack multipart archives. In this case you’ll have no choice but to obtain the original RAR for Linux software by enabling Multiverse repository. Alternatively, you can download the trial version directly from the official RARLAB site.

Posted in HOWTO | Tagged: , , | Leave a Comment »