Reading:
Making The Most Of Your Python Script: Update, Reuse And Maintain
Share:

Making The Most Of Your Python Script: Update, Reuse And Maintain

How To Use Python To Automate Almost Anything: Article 3 of 3

Content in review

In part 2 of this series, I outlined writing a script in Python which was an example real-world task a tester may face; taking screenshots of websites based on a list of URLs. The URL check script demonstrated some basic aspects of Python, such as looping, file input and output, and how to work with third-party libraries. The script was a good start but let’s make it a bit more helpful by adding some additional capabilities.

Thinking about how a script may be used and reused can really help with the overall benefit of a script. Sometimes a script may be written for single use, but later on there is a need for the script to be reusable. Thinking ahead and about how a script can be reused can avoid messy situations where a script is used in ways it wasn’t intended to.

The Script, Revisited

As a reminder, here’s the script for capturing screenshots of URLs from part 2:

import os from selenium import webdriver if not os.path.exists('shots'):   os.makedirs('shots') with open('url_list.txt') as f:   urls = f.readlines() driver = webdriver.Chrome(‘/path/to/chromedriver/’) count = 1 for url in urls:   url = url.strip()   driver.get(url)   driver.save_screenshot('shots/screenshot{}.png'.format(count))   count = count + 1 driver.close()

To run this script, first make sure that the file url_list.txt is located in the same directory as the script, then open a terminal and run:

python take_screenshot.py

This script loops through the list of URLs and for each one, a browser visits the URL, takes a screenshot and saves this screenshot to a file. This works pretty well, but it does have a fairly serious shortcoming: this script only works properly if the file containing the list of URLs is located in the same place as the script itself. What happens if tests need to be created in a different location with different files? One solution is to copy this script to new locations and run it in place, but that’s a short term hack. If this script ever needs to change, all copies would have to change or be replaced. There has to be a better way!

One obvious solution is to provide a target location for the file containing the URL instead of assuming the file is located in the same directory as the script. Let’s see what this could look like.

Using Script Variables

Suppose we want to set the directory to a specific directory instead of the current directory. This means modifying the lines below:

with open('url_list.txt') as f:   urls = f.readlines()

This line opens a file called url_list.txt  in the same location as the script and assigns the file object to the variable f. Using this file object, each line from the file is read and added as an entry in a list object called urls. Improving the code and making it more flexible can be done in several ways.

Adding A Target Path

Taking the advice of trying the simplest thing that could possibly work, this path could be hardcoded in the script. Putting a target path name to the url list file into a variable and passing that variable into the open() is pretty simple, so that’s the first approach to try.

Adding in a variable representing the target directory for looping over looks like this:

input_list = 'full/path/to/file/' with open(input_list) as f:   urls = f.readlines()

This is a pretty simple approach that makes this script more maintainable. This also shows one of the nice things about Python for writing small scripts: it’s easy to add variables “on the fly” where needed in a script.

Adding A Target Output Path

While changing the original script to add a target path, you’ll notice we’re still generating a directory called shots in the place that the script is located. Similar to allowing the url list file to be located somewhere arbitrary, we can place the location of the shots directory into a variable. Doing this makes the part of the script that creates the directory for the screenshots look like this:

screenshots_directory = 'path/to/directory' if not os.path.exists(screenshots_directory):   os.makedirs(screenshots_directory)

To keep things consistent, we also have to edit the line inside the for-loop that saves the screenshot to the file. Recall the for-loop from our script looks like:

for url in urls:   url = url.strip()   driver.get(url)   driver.save_screenshot('shots/screenshot{}.png'.format(count))   count = count + 1

Using the newly created screenshots_directory variable in the for-loop makes the loop look like this:

for url in urls:   url = url.strip()   driver.get(url)   driver.save_screenshot('{}/screenshot{}.png'.format(screenshots_directory, count))   count = count + 1

Here, we see that the format function can take multiple values and insert them into a string in the same way as a single value. Each {} gets replaced with the values passed into format in the order they’re passed in.

The advantage to this approach is its simplicity. Adding in variables adds quite a bit of reuse for the URL screenshot script. These variables can be stored and updated as needed, depending on the scenario.The downside to this approach is that the url list file location and screenshot directory are now directly tied to these hard-coded values. These values have to be found and changed each time a new set of screenshots needs to be created. It may be better to somehow dynamically input these values as required.

This is possible to do with Python using arguments passed into the script.

Using Script Arguments

Although adding in variables containing the url list file and screenshot directory is a good idea, it could be made better by using script arguments.

Instead of running the script on the command line as before, like this:

python take_screenshot.py

The script is called with additional values on the command line like this:

python take_screenshot.py /path/to/list /path/to/screenshots/dir/

The locations of the url list and screenshot directories can be chosen by the user when the script is run instead of hardcoding the values in the script. This is a big help towards making the script more practical and reusable.

Argparse Library

Fortunately, there’s a library in Python that can help with this task called argparse. This library provides functions for parsing additional arguments to a Python script. It’s also part of the default libraries that are included with Python installations.

To add this library to our URL script, add the statement to the top of the script:

import argparse

Now, all the capabilities of the argparse library are available to use throughout the script.

The most basic use for argparse is to collect arguments that are included when calling a script and placing them into an object where they can be used. There’s much more functionality present in argparse, but for now, the focus will be on this basic use.

Now, both the url list file and screenshots directory can take a value from a script argument.

Assigning Values Via Arguments

Luckily, the variables introduced when hard-coding values are reusable here. The value taken from the command line arguments can be assigned to the variable we created for hard-coding the values, making use of some previous work we have done.

Assigning the value from the command line arguments to the target directory variable, using argparse, looks like this:

parser = argparse.ArgumentParser() parser.add_argument("url_list") parser.add_argument("screenshots_directory") args = parser.parse_args()

For completeness, let’s walk through what these lines do.

Explaining Argparse Additions

These lines create an ArgumentParser object. It looks for two arguments in the command line called “url_list” and “screenshots_directory”, respectively, parses the arguments passed into the script from the command line and stores them in an object called args.

We can then pull out the values passed from the command line representing the url list location and screenshot directories, and places them into variables at the top of the script like this:

url_list = args.url_list screenshot_directory = args.screenshot_directory

It’s a few lines of code that help out quite a bit!

Completed Script Example

The whole script now looks like this:

import os import argparse From selenium import webdriver parser = argparse.ArgumentParser() parser.add_argument("url_list") parser.add_argument("screenshots_directory") args = parser.parse_args() url_list = args.url_list screenshot_directory = args.screenshot_directory if not os.path.exists(screenshot_directory):   os.makedirs(screenshot_directory) with open(url_list) as f:   urls = f.readlines() driver = webdriver.Chrome(‘/path/to/chromedriver/’) count = 1 for url in urls:   url = url.strip()   driver.get(url)   driver.save_screenshot('{}/screenshot{}.png'.format(count, screenshot_directory))   count = count + 1 driver.close()

This script can now be called with an argument like this:

python take_screenshot.py /path/to/list /path/to/screenshots/dir/

Improving On Argparse: Adding Error Handling

Things are looking pretty good! Starting with an improvement of hardcoding values turned into being able to choose the location of the url list and screenshot directory each time the script is run. This makes this utility script much more flexible and helpful for users. The argparse library also handles some basic command line usage situations, which we can add into the script as well.

Argparse Error Handling

Suppose this script is shared among team members and someone who hasn’t used it before wants to use it to create some test templates in their project. What happens if they forget to specify one of the variables? Or mix up the order?

Running the original command:

python take_screenshot.py

Produces an error message as following:

usage: parse.py [-h] directory parse.py: error: the following arguments are required: url_list, screenshot_directory

This is sort of helpful since it tells the user parameters are missing. It also gives some clue for using a help command (-h) to find out more information. However, it doesn’t really tell us how to resolve this issue. We want a useful message that would point out to the user that they are missing the directory argument.

Custom Help Messages

Argparse does allow custom help messages to be added fairly easily to the parser object. This means that if an error is shown because of missing or bad arguments, the user can call a “help” command to see what the correct argument should be.

Adding a helpful message describing that the directory argument needs to be a full path to a target directory might look like this:

parser = argparse.ArgumentParser() parser.add_argument("url_list", "Location of the file of the list of URLs") parser.add_argument("screenshots_directory",   "Location of the resulting screenshot files created") args = parser.parse_args()

The only change we need to make is to add a help string to the parser.add_argument() call. Adding this in, calling the script with a --help option:

python take_screenshots.py --help

gives the more helpful output:

usage: parse.py [-h] directory positional arguments: url_list                Location of the file of the list of URLs screenshots_directory   Location of the resulting screenshot files created optional arguments: -h, --help  show this help message and exit

More information is often better when getting started with a new tool, and there’s now at least some basic documentation around how to use this script.

Argparse can do even more complex operations with arguments which may also be beneficial in the future.

What We’ve Learned So Far

Looking at this series of articles, we’ve seen how to setup Python to write simple scripts.

In part 1, we walked through setting up everything that’s needed to get started with Python. Some key ideas from this article:

  • Python is a good choice for testers as a tool for automating simple tasks.
  • Python is cross-platform (works on Windows, Mac or Linux) but requires slightly different setup in each case.
  • A basic introduction to what comes with Python “out-of-the-box”.

In part 2, we identified a situation where writing a Python script would be helpful for a tester in creating unit test templates for a project that was lacking one. This provided a concrete case of using Python, which included some these key ideas:

  • One approach to solving a testing-related task using a script for automation
  • What a basic Python script file looks like and how it can be run
  • How to do basic scripting operations in Python such as opening files and writing conditional statements and loops
  • What a finished script in Python looks like combining these operations together

In this article, we looked at improving this script, making it easier to use multiple times by allowing a target directory to be chosen at the time the script is run. Here some more high-level ideas around Python were introduced, such as these key ideas:

  • Updating an existing script might be helpful to make it more reusable and maintainable
  • Describing one simple approach to specifying values that may need to change - in this case, a target directory path being used as a variable
  • Using third-party modules for specific tasks (in this case, using argparse for command-line option handling)
  • Using argparse to help choose a target directory at runtime instead of internally in the script.

You've now seen enough Python to be able to write many other robust scripts that you can use to automate daily tasks that all testers face. Why not share your automation progress and ideas in The Club?

References

Josh Grant's profile
Josh Grant

Josh Grant is interested in automation, software development and big ideas. He’s currently working as a test developer doing cool automation work in Toronto, Canada. You can find him active on Twitter.



Unified Cross Platform and Application Testing by Keysight
Should We Just... Delete It? - João Proença
When to Say No to Automation - Jack Taylor
Five Optimization And Performance Tools That Enhance Website User Experience
Jumpstarting Your API Test Automation: Which Test Creation Tool Is Easiest To Learn?
Making Browser Automation Easy With Python and WebDriver
Explore MoT
TestBash Brighton 2024
Thu, 12 Sep 2024, 9:00 AM
We’re shaking things up and bringing TestBash back to Brighton on September 12th and 13th, 2024.
MoT Foundation Certificate in Test Automation
Unlock the essential skills to transition into Test Automation through interactive, community-driven learning, backed by industry expertise