5 Reasons Open Source Software is Good For Your Business

Open Source Initiative Logo

In recent years, open source software has become more frequently used by businesses and individuals alike. Why is this, and what makes open source solutions so increasingly popular? Below I list five reasons why open source software can be good for your business.

Security

The security of software used in your business is very important. It is used to protect, process and store your company’s intellectual property. By its nature open source software has the code it comprises of visible to all. This means large open source projects are frequently reviewed by the community, allowing security vulnerability to be spotted and resolved quickly and effectively. Compare this to proprietary software. With programs that do not have the code publicly available, you must rely upon the vendor to identify security issues. Not only that, but you must then trust that these problems will be resolved (and distributed) in a timely manner – a task which often rubs against the vendor’s motives for profit.

Quality

Open source software is frequently made by people that have a real drive and interest in the software they are working on. They strive to create quality software and work with others to do the same. Additionally, open source software tends to meet the needs of its users very well. This is sometimes due simply to good initial design. However, a lot of the quality present in open source software is the result of users participating in its design and development. This can be as simple as reporting bugs or suggesting features or can be as involved as having meetings regarding the program’s design or even digging into the code itself. With proprietary solutions, the need for a feature and the amount of time dedicated to its quality development is mostly the function of the vendor’s or their clients’ budget. Improvement in the quality of proprietary code is otherwise driven only by market competition/demand and public relations.

Freedom

Many proprietary software packages will create lock-in for their users. This can be done by using custom protocols, undocumented file formats or otherwise not complying with established standards and best practices. This ensures users and companies that use the vendor’s software will be forced into upgrades, new versions and compatible software in order to continue accessing their data. Such vendors often dictate their own vision of their software which may conflict with your business practices or requirements. Additional hardware may also be required to run the latest and greatest version of their software package. Therefore, as a business, you may often be at the mercy of your proprietary software vendor(s). In contrast to this, open source systems allow users to be in control of many of these decisions. It allows them to do what they want with the software. Not only is the code available, but with any significant open source project, there is typically a large community of developers and users willing to help with this.

Interoperability

As open source projects generally have no need to use proprietary file formats, they are generally more compliant with official standards. This can make them more interoperable with other business software. Standard file formats and network protocols enable your company to integrate open source solutions into a variety of applications. This is much more possible than with proprietary software, which can often encumber businesses and cause their software environments/infrastructure to be restricted by interoperability constraints.

Cost

Proprietary solutions typically force end-users and business owners to incur expensive licensing fees, often per site or even per machine. With vendor lock-in in effect, this can result in a high initial outlay as well as potentially prohibiting growth of the company’s information technology resources. Open source solutions are typically free to use. They also allow you to metaphorically ‘try-before-you-buy’. A company can try out a possible solution with no worry of directly incurred costs. If the company is able and willing they are also able to make modifications to the open source software they use, in order to meet business requirements, rather than be at the mercy of their vendor to implement a required feature.

Conclusion

Of course, all this being said, your business should not necessarily use open source software for everything. However, with all the benefits open source solutions and its community can offer, it would be neglectful to not seriously consider use open source solutions in at your company.

This post was originally written for the Rapid Web blog.

The Fundamentals of Git – Presentation

Git logo

I recently gave a presentation at Rapid Web regarding Git. It covers the background of Git, how it works internally, and some examples of daily use. The presentation also covers commonly used Git terminology and provides a frequently used command reference.

‘The Fundamentals of Git’ presentation is embedded below. Comments and constructive feedback are definitely welcome. This presentation is licensed under the Creative Commons Attribution-ShareAlike License.

The Fundamentals of Git Presentation – Description

Git is a distributed version control system, created by Linus Torvalds, and used worldwide by many software developers for a large number of well known software projects.

In this presentation, I explain the fundamentals of the Git software and some of its internal workings and related terminology. The target audience is individuals who have used Git before in a limited capacity and want expand their knowledge of the software to improve their daily use of it. The presentation could also be useful for those with knowledge of Git and a desire to improve their understanding of some of the more technically elements.

The Fundamentals of Git presentation is shared on slideshare.net.

How to generate test data for a web application

Fake personal details for unit tests

I’ve coded my fair share of websites that require user authentications in some form or another. Often, you’ll need to test registration forms, user details, billing/delivery address, foreign characters and all kinds of fluff related to user accounts.

Most of the time, this will be ‘user test’, ‘test1’, ‘test2’, ‘aaa’, ‘foo’, ‘123’. Sound familiar?

Sometimes you need believable information, either for testing purposes (fuzzy unit tests anyone?) or to show as a placeholder for real data on a website design. I’ve found the Faker PHP library is perfect for generating this kind of data.

Faker can generate a wide variety of testing data for your web application. This includes:

  • names
  • addresses
  • telephone numbers
  • companies
  • email addresses
  • websites
  • lorem ipsum
  • images
  • catch phrases
  • credit/debit card details
  • date times
  • user agents
  • colours

Its very easy to use as well. At its simplest, just include the library, instantiate its main class, and access its sensibly named attributes. It returns a string result as you’d expect.

require_once '/path/to/Faker/src/autoload.php';
$faker = Faker\Factory::create();

echo $faker->name;
echo $faker->address;
echo $faker->text; // lorem ipsum text

// Generate multiple different names
for ($i=0; $i < 10; $i++)
{
    echo $faker->name, "\n";
}

I’ve found this very useful. Take a look at the Faker GitHub pages.

PHP Optimisation Tricks & Tips

PHP optimisation stop watch

PHP may not be well known for being a fast and super efficient language, however there are plenty of tips and you can learn to make your heavily looped code somewhat more speedy.

Sometimes how to code your PHP in an optimised manner is be obvious, while other times you can be uncertain. In many cases, it can be truly unexpected which method of coding certain functionality is actually the most efficient.

I recently found the following very interesting article and PHP benchmarking tool. Take a look  for optimisations tricks and benchmarking of common PHP code snippets.

  • PHP Optimisation Tricks – This article describes a bunch of smaller tricks that can help make code run a little faster. In many cases, this can be useful when code is running many, many times within a loop. Please be aware that although some of these optimisation speed up the running of your code, some of them are unsightly and no where near as readable that more standard ways of coding. Some of the more bizarre optimisations should therefore be used sparingly and one could say, only when optimisation is needed.
  • The PHP Benchmark – phpbench.com runs many different code snippets in large loops. This allows visitors to easily compare the efficient of certain ways of coding the same type of functionality.  This is very interesting as a simple comparison, but it is worth being aware that due to PHP garbage collection and Zend optimisations results can vary. It is worth refreshing several times to see how/if the results change.

Java Robot for Automation

The Java Robot class contains many methods which can be useful for automation. This can be automation of testing procedures or alternatively operating existing programs via keystrokes and/or mouse presses. This can be useful if a program does not expose any form of interface or API that can be accessed programmatically.

The Java Robot class allows you to write programs that:

  1. Take a screenshot of the entire screen
  2. Take a screenshot of a specific rectangular area
  3. Move the mouse pointer to specific X, Y coordinates
  4. Perform standard mouse functions, like left click, middle click, right click, drag and drop and using the scroll wheel
  5. Perform standard keyboard functions, such as pressing individual keys and performing complex key combinations

For more practical applications, I previously wrote an article on automation using the Java Robot class which covered using the Java Robot class for automation of:

  1. Batch processing of files using a industry standard application
  2. Defeating 2D ‘Game’ CAPTCHAs – In this case, I was testing against the Are You human PlayThru demo CAPTCHA from http://areyouahuman.com/demo
  3. Tracking and following an object in a 3D game world – I used Minecraft as the example here, and make a very simply bot using the Java Robot class that would identify and head towards any yellow flowers shown on the screen.

 

Using Exceptions and Try Catch Blocks in PHP

Although common place in most languages like C, C++, Java and so on, Exceptions and the use of Try Catch blocks are no where near as prevalent in the PHP code I’ve encountered.

From now on, I intend to use them much more in the PHP functions and class methods I write, rather than following the seemingly common practice of having these return null, false or -1.

Here’s an example of how Exceptions can neaten up code. This is a factory class that creates a new car object based on information retrieved from a database. I’m using this mysqli wrapper in this example.

abstract class car_factory

{

public function get($car_id)

{

$db = db::get();

if (!$db) throw new Exception(“Database connection failed.”);

$rows = $db->q(“select name, description where id = ? limit 1”, “i”, $car_id);

if (!$rows) throw new Exception(“Database query failed.”);

$row = $rows[0];

return new car($row[“id”], $row[“name”], $row[“description”]);

}

}

And below is how we’d call this get method.

try

{

car_factory::get(1)

}

catch (Exception $e)

{

echo “Dreadfully sorry, but an error occurred. Here are the details: “.$e->getMessage();

}

For those unfamilar with Try Catch blocks, this code will attempt to perform what is within the ‘try’ brackets. If an exception is thrown at any point during this, it will abort and run the code within the ‘catch’ brackets, creating a new PHP Exception object, containing the specific details of the exception that was thrown.

This Exception objects contain a lot of methods that can be useful for error reporting and/or logging. Click the links in the list below to see the specific details on PHP.net.

Using Exceptions, rather than returning null/false, or relying of PHP’s warnings, errors and notices, can generally result in easier management of errors.

For example, the above example could be easily modified to behave in a different manner dependent on whether or not the website is in a live or development status, which could simply be determined by a global ‘$site_live’ boolean variable in one of the website’s main configuration files.

Automation with the Java Robot Class

Java logo

I’ve recently been experimenting with Java Robot class, which allows one to control mouse pointer position, clicks and emulate keyboard input. This makes the Java Robot class perfect for implementing automation in systems which have no accessible API.

The Robot class can also be used to take screenshots of all or part of the screen, allowing automated tasks to be visually verified. For example, a program designed to open a specific file in a basic image editor could take to the following steps:

  1. Press CTRL+O, the keyboard shortcut typically used to access the ‘Open File’ window.
  2. Wait 1 second for file open window to appear.
  3. Type the file name ‘test.jpg’.
  4. Press Enter.
  5. Wait 2 seconds for file to open.
  6. etc.

In the above example, delays are used between keyboard input actions, in an attempt to make sure the application the robot is interacting with is ‘keeping up’.

This works, but could be unreliable if the application we are interacting with is delayed for any reason. There are many reasons why this may be the case, such as background tasks running (antivirus, operating system updates, etc.). As well as this, the application we are interacting with may behave differently than expected. For example, the application could show a dialog box that was not expected. Using the current program, the robot would continue typing regardless. This is the equivalent of a human attempting to use a computer normally, whilst at the same time, being blindfolded.

To get around this, we can use the screenshot ability of the Robot class to visually verify that the results of our keyboard or mouse actions are as expected. The example below shows how we could re-work the previous set of instructions, to utilise visual verification.

  1. Press CTRL+O, the keyboard shortcut typically used to access the ‘Open File’ window.
  2. Take a screenshot and search the screen for the title bar of the expected ‘Open File’ window. Repeat until found.
  3. Type the file name ‘test.jpg’.
  4. Press Enter.
  5. Take a screenshot and search the screen for the expected image that we just asked the image editor to open. Repeat until found.
  6. etc.

This method not only visually verifies what we expected actually happened, but also works to speed up the program. If the loading of the image took less than the ‘2 seconds’ in the first example, the program would still wait regardless. In the second example, the robot would continue with its sequence of events as soon as it determined the image was loaded (and being shown in the image editor’s window).

I’ve recently become quite interest in automation via this class and have written a number of application’s to do various tasks, from playing basic games to automating the operating of a complex industry standard application.

Some of what I have with these ideas is listed below.

Automated batch processing of files using an industry standard application

I am not currently able to discuss the precise details of the application this robot uses and thus this description is very vague, but I can say that this application of the Robot class does the following.

  1. Retrieves task information from a central database-driven API that holds a queue of task to complete
  2. Downloads the prerequisite file required to complete the task
  3. Emulates the pressing of keyboard shortcuts to make the application it interacts with complete the task requested, and save out the required results.
  4. Submits the results to the central API, which then marks that specific task as complete.

In this implementation, I wrote the server side code and the client side robot to work directly with one another to automate the batch production of these ‘results’. This results are therefore generated at speeds much faster than any human could operate the application being used, via the rapid, automated use of keyboard shortcuts.

I’ll post again about this when I’m permitted to add additional details.

Defeating basic 2D games

Using the ‘Are you Human?’ game based CAPTCHAs as a test bed, I have successfully created a robot that can defeat the majority of the small games available on the demo page at http://areyouahuman.com/demo.

The program scans the screen for instructions to determine if a game it is capable of playing is shown and when found, will click the start button and follow a sequence of steps to play and win that particular game type.

The majority of these games revolve around dragging moving objects to a destination object, such as dragging images of food to a fridge. This is completed in two steps.

The first step is scanning the screen for a known image that need dragging, moving the mouse cursor to it gradually and pressing the left mouse button. In the second step, the program scans the screen for the image of the destination area, and when found gradually moves the move cursor to it, and releases the left mouse button. These two steps, repeated for all known images we need to drag, results in easily winning the game.

Due to the nature of these games being used as CAPTCHAs, I will not be released this code to anyone so please do not ask. Unless the guys at ‘are you human’ wish to use it to help improve their system, in which case I will be happy to provide them with the working program.

It is worth noting that after working on this as a programming challenge for myself, I discovered someone else also had the same idea. An article at Spamtech discusses cracking the Are you human CAPTCHAs, this time in Python using existing computer vision libraries.

Tracking and following targets in a 3D world

This was another challenge I set myself. Being a fan of Minecraft, put very simply, a first-person 3D block building game, I thought it may be interested to see how easy it would be to make an automated bot that would identify an object, aim the centre screen cross hair at it and then walk towards it.

I chose the common ‘yellow flower’ of Minecraft as my target object and proceeded to write the code to identify this object on the screen.

Due to the nature of 3D perspective, it was not possible to simply scan the screen for previously captured images of the yellow flower, and its size and orientation differed dependent of how far away the object was from the player and which way the player was facing. To work around this issues, I chose to simply identify the yellow flower by searching for pixels on the screen within a certain RGB range that matched the colour of the yellow flower within the game’s day and night cycles.

After identifying the flower, the robot needed to aim at the flower. The 3D nature of the game again complicates things here, as a single movement to slightly re-aim modifies the entire view of the world from the first person perspective. I coded this such that when the flower was identified, its location on screen would be compared to the location of the cross hair. The mouse would then be moved a small number of pixels towards this location, thus moving the 3D view slightly closer to the target. Because this slightly movement has caused the view to change, the flower’s location needs to be identified again.

The process of identifying the flower and moving the mouse cursor slightly towards it is repeated continually, until the yellow flower is identified as being within an acceptable range of the cross hair. It can then be said the flower is directly in front of the player. At this point, ‘W’ is held for 1 second (to move foward), released, and then the entire process repeats to keep tracking and moving towards the flower.

The entire process looks like this:

  1. Identify position of the yellow flower on the screen based on appropriate colour.
  2. If the yellow flower is close to the cross hair, and thus is directly in front of the player, move forward for 1 second.
  3. If the yellow flower is not close to the cross hair, move the mouse pointer slightly in the direction of the flower to adjust to view point.
  4. Go to step 1.

The only other part of this little program was a small piece of code to handle a situation where no appropriate yellow was detected. I handled this simply by having the program move the mouse right slowly, by a small number of pixels in order to turn the player’s view right, in the hope of finding something yellow.

It is worth noting that my recognition of the yellow flower simply by a RGB range similar to the colour of the flower turned out to be insufficient on its own. Left to its own devices in a standard Minecraft world, I found the robot not only chasing down yellow flowers, but also being drawn to the orange-yellowish of torch flames.

In addition, it attempted to commit suicide several times by attempting to jump into pits of glowing  red/yellow lava!

Bitcoin Helper 0.11 – Error handling and caching

Warning iconThis is a quick post to announce the release of Bitcoin Helper 0.11. Bitcoin Helper 0.11 contains significantly better error handler than the initial release (0.1) and also makes the file-based caching of the JSON retrieved from bitcoincharts.com mandatory.

Error Handling

It can now handle internal issues with network communication to the data source and incorrect JSON formatting of retrieved data. Bitcoin Helper 0.11 also checks to ensure the requested ISO 4217 currency code, for conversion to Bitcoin, is supported.

Certain functions in Bitcoin Helper now return a negative integer in the case that an error occurs. Details regarding these error codes can be found in the source code comments of bitcoin_helper.php. The relevant section of commenting is quoted below for convenience.

/* Information about error codes

Certain functions within Bitcoin Helper (0.11 onwards) will return
numeric error codes in the case of unexpected error or failure.

You should make sure to check the returned value for error codes
(intergers less than zero) in your code. The following reference
shows the error numbers and an explanation.

* -1 = Network error retrieving data from bitcoincharts.com
* -2 = Error decoding JSON data retrieved from bitcoincharts.com
* -3 = Currency code not supported
* -4 = Could not write to cache file – check permissions!
*/

File-based Caching

Bitcoin Helper 0.1 would cache the JSON containing currency exchange rates to a file if possible. This caching is now a requirement as of Bitcoin Helper 0.11.

Bitcoin Helper will attempt to write a file named ‘bitcoin_weighted_prices.json’. If this file is not writeable, you will receive error code -4.

If you receive this error, you will need to alter the permissions of the directory to allow this file to be created. Once created, you can revert the directory permissions if you desire and make only the individual ‘bitcoin_weighted_prices.json’ file writeable.

Download Bitcoin Helper 0.11

To download Bitcoin Helper 0.11 with improved error handling, please go to the Bitcoin Helper project page.

Bitcoin Helper 0.1 – Converting currencies to Bitcoin

Over the past few months, I’ve gained quite an interest in the crypto-currency Bitcoin.

Bitcoin is a new currency which has no physical form, but can be considered Internet cash. If you want more information on Bitcoin in general, take a look at the We Use Coins website, which has a great introductory video and articles/links to other useful Bitcoin resources.

Bitcoin Helper 0.1 Released

As part of my interest in Bitcoin, I’ve decided to start a mini-project in the form of Bitcoin Helper, a PHP class which will contain multiple methods to aid web site/application developers in integrating Bitcoin related functionality into their systems. The first function, which comes in Bitcoin Helper 0.1, is a really easy to use converter that can convert non Bitcoin currencies into the equivalent Bitcoin value and return the result.

To see how simple it is to use, see the following code snippet.

include "bitcoin_helper.php";
$bh = new bitcoin_helper;
echo $bh->convert_to_btc("GBP", 10);

This will convert £10 (Great British Pounds) into Bitcoin and echo out the result. The following works for USD and Euros as well.

echo $bh->convert_to_btc("USD", 10);
echo $bh->convert_to_btc("EUR", 10);

The trade data for several other currencies is provided from the external source (see the Bitcoin Charts website). If your currency is listed there, entering its ISO 4217 currency code as the first parameter of the convert_to_btc function should allow you to convert any value of that currency into Bitcoin.

Please note that this is the first release of Bitcoin Helper, so I can not guarantee it is bug free. More checks, validation, improvements and new features will be added in the future versions. Feel free to leave a comment with any suggestions.

I hope this very basic Bitcoin Helper release can assist web developers with some simple Bitcoin conversion in their sites.

Download Bitcoin Helper 0.1

To download this first version of Bitcoin Helper (0.1), please visit the Bitcoin Helper project page.

Program Design – “90% of your users are idiots”

They are. Or at least, it can really help to assume they are.

A short while ago, I read this from Alex Yumashev’s Jitbit blog:

I just overheard this conversation between two developers at a co-working site:

“I plan [on] creating a prototype for my new XXXX application, whatcha think it should be – a web-app, or a desktop app?”

The answer was:

“90% of your users are idiots who won’t be able to tell the difference”

I think I just found my answer to the ultimate question of life universe and everything and it’s not “42”.

It’s “90% of your users are idiots”.

Now thats a questionable term to describe your customers. I don’t think 90% of my users are idiots. But. That’s the way to think of your users when making design decisions and building your interface.

Jitbit Blog

And he’s very right.

Thinking about some of the largest and most successful software developments, this rule applies – not that 90% of the users of these systems are idiots, but more that they are designed to accommodative for 90% of their users being idiots .

Think about the operating systems environment, in which Mac OS X and the Windows series of operating systems are by far the most popular on the desktop. Both of these systems are developed to heavily guide the user through any process or task that he or she wishes to complete, metaphorically holding the users hand throughout each step.

The mobile market is the same. Up until very recently, iOS (the Apple operating system powering the iPad and iPhone) was very far in the lead in terms of sales and end-user popularity. Android has only recently, in the past few months starting competing well with iOS. iOS is simple. In my opinion, too simple, but people like it and use it, on a huge scale. 90% of users are idiots. As Android has got more and more user-friendly, it has increased in popularity amongst end-users, along with the ever increasing push from non-Apple phone and tablet manufacturers to use Android.

Both user interface and structural application decisions can often benefit from thinking that 90% of your users are idiots.

  1. Should I implement feature X in either A or B way? Whichever is easier for those 90%.
  2. Should I design the user interface to promote feature A or B most promotion? Whichever is most appropriate for those 90%.
  3. Do I design this icon that triggers feature X to represent concept A or concept B? Whichever will make the most sense to those 90% of users.
  4. Should I document this little used and/or somewhat obvious feature? Of course – 90% of your users are idiots.

The experts among us do not necessary need or want to be guided though every step of a process with our hands held all the time. Some of us just want to dig into plain text config files and command line interfaces. The developers and the computing enthusiasts, we’re the remaining 10%.