GIS 5103 Module 11: Sharing Tools

The eleventh and final module of the course taught us the correct procedure to create easier to share tools.  The ability to share script tools with others is one of their key benefits, yet it is important to provide necessary documentation as well as the correct file structure to ensure it will run properly.

For our assignment, we were provided with a script, script tool and toolbox.  While the script tool parameters were preset, we were required to edit the script itself to accept user inputs for two of the parameters.  Because we were also instructed to rename the provided files, the script tool properties needed to be updated in order to locate the script.  We also needed to edit the script tool's item description for each parameter in ArcCatalog.  This significantly improves the user interface for those not familiar with the tool, as the description we entered would be displayed in the help window of the script tool's dialog box.

Finally, we embedded the script into the tool and set up a password.  This allows the tool to be shared more easily by dispensing with the need for the stand-alone script.  It also increases security on the tool by preventing unauthorized users from altering the script.

GIS 5103 Participation Post #2: Viewshed Analysis in Archaeological Modeling

In their 2014 article "A house with a view? Multi-modal inference, visibility fields, and point process analysis of a Bronze Age settlement on Leskernick Hill (Cornwall, UK)" (2014, Journal of Archaeological Science 43, 267-277), Stuart Eve and Enrico Crema describe their efforts to model Bronze Age sites using different sets of variables and critique the way in which some researchers approach statistical modelling.  Modelling, as the authors use the term, involves performing spatial analyses on variables that potentially affected site placement; a successful model would reveal correlations between variable values and site placement.  For example, a model might propose that sites are more likely to be located at certain elevations or in areas of a certain level of precipitation.  Environmental variables such as elevation, slope, aspect, land cover, rainfall, and distance to permanent water sources are all common variables to consider when constructing a model for analysis.  Eve and Crema emphasize that no model is "true"; there are only more or less successful models.  Even a successful model, showing strong statistical correlation between variables and site placement, is not guaranteed to be closer to true than other models.  A wide range of variables and different levels of analyses must be considered.

Eve and Crema investigate three proposed models for the location of sites on Leskernick Hill.  The two models unique to their study involve line of sight analyses.  The first model proposed the placement of sites to maintain line of sight with ritual sites in the area while the second proposed a line of sight preference for tin deposits.  The third model looked at the more standard topographic and environmental variables.  The line of sight analyses were created by calculating the viewshed of every raster cell coinciding with Leskernick Hill using GRASS GIS and Python for batch processing.

The results indicate the need to consider difference scales of analyses.  While the first model (line of sight with ritual sites) was the best match overall, sites in the western portion of the study area fit the second model (line of sight to tin resource) much better.  Removing these western sites also showed the southern sites fit the first model even better.  The topographic inputs of the third model did not fit site location very well.

Link to article (hopefully one that works this time):

10.1016/j.jas.2013.12.019

GIS 5103 Module 10: Creating Custom Tools

Script tool messages after replacing print with arcpy.AddMessage().

Dialog box of script tool parameters.

While Python scripting for ArcMap allows one much more freedom to create functions not otherwise available, using and editing Python script requires a degree of knowledge and experience.  Fortunately, it is possible to import a stand-alone Python script into a toolbox as a script tool that is more easily shared with and used by others, regardless of their knowledge of Python.  Our exercise and assignment this week taught us to create and edit such script tools.

Although creating a script tool is a relatively simple matter, it does require a few steps to ensure the tool works properly without further editing the script itself.  After importing the script as a script tool into a toolbox, one must set up the properties and parameters of the tool.  The parameters include all the variables for the tool (inputs, outputs, etc.) as well as their data types.  This sets up the dialog box within the ArcMap GUI, but the Python code itself must be edited to accord with these parameters to allow users to successful enter variables into the tool.  This is accomplished by using the GetParameter or GetParameterAsText functions.  These are numbered in parentheses in the order they appear in the dialog box, starting with zero.  This will allow the script tool to function with custom inputs from the user.  The only remaining task is to use the AddMessage function rather than print to display the tool's messages in the results window.

GIS 5103 Module 9: Debugging and Error Handling

This week's module formally introduced us to the tools and tasks involved in debugging scripts.  While I'm sure we have all been forced to debug our scripts throughout the course, this module taught us how best to handle the inevitable errors that will plague us in our careers.  The debugging tool in PythonWin and try-except statements in particular will be useful in the future.

This assignment differed from previous ones in that we were given three scripts and tasked with debugging them.  The first two scripts contained several errors we needed to locate and correct for them to run properly.  For the final script, we were to insert a try-except statement that would trap the error in part A, print the error message to the interactive window, and continue to run part B of the script.  Below are screenshots of the interactive window messages resulting from my corrected scripts.

Script 1 interactive window after debugging.

Script 2 interactive window after debugging.

Script 3 interactive window after adding try-except statement.

GIS 5103 Module 8: Working with Rasters

This week saw us experiment with manipulating raster data using Python script.  As we did previously with vector data, we learned how to return information on raster files, such as spatial reference and cell size.  Because rasters are often large files, Python script will reference a raster object and create temporary rasters rather than create multiple raster files.  This is efficient, but it does require one to remember to save the final raster, otherwise nothing will remain after the script is complete.  

As in previous modules, our assignment required us to utilize many of the tools learned about in the exercise and expand them a bit further.  After creating a for loop to ensure the Spatial Analyst extension was available, our script needed to go through several steps.  First, it needed to use the RemapValue and Reclassify functions to collapse three separate classification of the landcover raster to one.  It then needed to create temporary rasters that would be used to create a final output between specific ranges of slope and aspect.  To do this, we needed to create four temporary rasters.  Two of the rasters were to define the slope range, with one including slope less than the maximum value while the other included slope above the minimum value.  The other two rasters would do the same for aspect.  These four rasters were to be combined, along with the reclassified landcover raster, using the "&" operator.  This would create a final raster with slope and elevation between the specified ranges along with the reclassified landcover data.  This final raster then needed to be saved in order to be permanent.  A screenshot of this raster can be seen above.  We were also required to insert print statements in our script to show in the interactive window the process of the script as it was run (see screenshot below).

GIS 5103 Module 7: Working with Geometries

The week saw us going further with the use of cursors and nested for loops.  Our exercise and assignment allowed us to explore getting coordinate data from points, lines, polygons and multipart features.  Although I was a little confused on the exact function of cursors when they were first introduced, this module helped me get a better understanding of their syntax and use.
An additional task required by the assignment was to create and write to a text file (see screenshot above).  This was covered in a previous chapter, so I am glad to get experience with editing text files in Python this week.  It is interesting to see how the syntax learned in previous modules is incorporated in new tasks each week.  I have even begun experimenting with Python in my internship, with varying levels of success.

Below is a copy of the pseudocode from my process summary.  Pseudocode is something I have neglected a bit throughout the course, but I can see how useful it would be as scripts increase in complexity.

Start

    import modules

    allow overwrite of previous data

    set workspace

    variable1 = "rivers.shp"

    set search cursor to retrieve variable1's "OID"  and "NAME" fields + full geometry object

    create/open writable text file "rivers_tevans.txt" in S:\GISProgramming\Module7\Data

   

    begin for loop for cursor:

                print "NAME" field

                set index variable/vertex ID

                for loop for point in row with .getPart function

                        add 1 to index variable

                        write to text file row's OID, Vertex ID, X coordinate, Y coordinate, and name

                        print OID, Vertex ID, X coordinate, Y coordinate, and name

  close text

  delete row and cursor

End

GIS 5103 Module 6: Exploring & Manipulating Spatial Data

This week's module took us deeper into working with GIS data using Python scripts.  An important initial task was to explore how to check for the existence of data files and, if they exist, to look at their attributes (datatype, for example).  This is something one can do without much thought through the ArcCatalog GUI, but Python also provides easy access if the correct syntax is known.  We then further explored the use and manipulation of lists, tuples, dictionaries, and cursors.  It took me some time to understand how cursors work; they provide access to the rows of GIS attribute tables.  Like lists, cursors can be used in loops to iterate through each row for particular tasks.  The last major task of our assignment was to populate a dictionary using a search cursor and for loop.  The cursor was required to return only certain rows that met the criteria of an SQL query and then return two particular fields from those rows.  The dictionary was then populated with the data from these two fields.  A finishing touch this week was to add print messages throughout the script to provide updates in the interactive window on the status of the operations being run (see screenshot below).  This can be useful with scripts such as this that can take some time to finish.  Without the messages, one will be left in the dark on the status of the script's run.  

This was a challenging assignment that took some time and thought to work out properly.  I appreciated the extra time provided for this module.  

GIS 5103 Module 5: Geoprocessing with Python

The whole point of this course is to learn how to use Python to geoprocess data using the tools of ArcGIS.  With module 5 we are now getting familiar enough with the syntax of Python to use it to accomplish geoprocessing tasks.  These tasks may be simple, but they will be the building blocks for more complex tasks in future modules.

Our assignment for the week was simple yet required some thought and planning to complete correctly.  Our script was required to add XY coordinates to a layer, create a 1000 meter buffer around the features of that layer, then create a new buffer layer dissolved into a single feature.  The standalone script also needed to print the messages of the tools in the Interactive Window (see screenshot above).  To run more than once, we also needed to include code to allow the overwriting of previous data.

To add XY coordinates to the hospitals layer, I consulted ArcGIS help on the proper syntax for the AddXY_management tool.  As described in the help file, I also used the Copy_management tool to first create a new copy of the hospitals shapefile to which was then appended XY coordinates with the AddXY tool.  I also used this tool to experiment with creating variables for the input parameters as described in the exercise.

The rest of the script was relatively simple to write.  The only issue I ran into was the failure of the script to overwrite previous data.  After some time fiddling with the script, I realized ArcMap was open in the background.  Closing ArcMap allowed the script to run and overwrite without error.

GIS 5103 Module 4: Python Fundamentals Part II

Module 4 continued our journey into the basic syntax of Python script.  The assignment this week was a little more complicated than last week.  This time we were given a piece of code composed of a for loop and an if-elif-else conditional that took the form of a dice game.  This code, however, contained two errors.  We had to debug the script by finding and correcting the errors.  Once the script ran without error, we then had to add new script that accomplished a couple of tasks.  First, the script needed to use a while loop to add 20 numbers to a list and then print that list.  The numbers had to be random and between 0 and 10.  The next, multi-step task was to remove a chosen number from the list and print a message detailing the number of times that number was removed.  This I accomplished using the count method (to find how many times the chosen number occurred in the list), an if-elif-else conditional (to determine which message to print) and a while loop (to remove each instance of that number).  The final task for the script was to print the updated list.

This was a challenging but very enjoyable assignment.  At one point (at least), I was confused on how to proceed.  Rather than dig through the text, I searched for relevant Python forums and tutorials for guidance.  As we have already learned, there appears to be a large community of helpful Python coders that have provided much useful information online.

GIS 5103 Module 3: Python Fundamentals Part 1

The screenshot above might not seem that impressive, yet it is the output of our Module 3 assignment for GIS Programming.  I am happy to get to this module, as it begins to introduce the building blocks of Python scripting.  The previous two modules saw us using scripts with syntax we did not understand.  Now, however, we are learning the details of Python scripting.  The structure of the module, providing examples in the exercise then requiring us to write a script with very little instruction in the assignment, works well to force us to think of the logic of the syntax and how to structure our script.

The assignment required us to write a script that created a string of our name, then to perform various actions on that string.  We had to split our full name into a list, then print our last name using indexing, then create a variable for the length of our last name, then multiply that value by three, and finally to print that value.  The only output of the script should be our last name and the final value.  It was a fun assignment that forced us to pay attention to the details of our syntax to avoid errors.  I look forward to learning more Python basics in module 4.

GIS 5103 Participation Assignment 1: GIS, Archaeology, and Human Movement Modeling

Fig. 2. The location of the Missaukee Earthworks (20MA11-12) and the twenty-nine contemporary, significant Late Prehistoric (AD 1200-1600) sites used as the focal node input for Circuitscape resistance analysis. (From Howey, 2011)

The article I read as an example of GIS used in the real world was “Multiple pathways across past landscapes: circuit theory as a complementary geospatial method to least cost path for modeling past movement” by Meghan C. L. Howey (Journal of Archaeological Science, 38 (2011), 2523-2535).  Howey uses the topic of population movement among the peoples of northern Michigan between 1200 and 1600 AD to test least cost path modeling and circuit modeling of this movement.  Specifically, she looks at the movement to and from the Missaukee Earthworks site, an important trade and ritual center.  Howey’s circuit model is run by an ArcGIS tool called Circuitscape.

The simplest version of least cost path modeling would consider only distance and topography, which, according to Howey, some researchers still do.  However, Howey also included water routes and land cover in her original least cost path analysis.  The output of a least cost path model is a single route that, based on the input parameters, represents the route of least resistance between two points.  A circuit model, on the other hand, considers the presence of multiple routes between two points in its analysis of the relative ease or difficulty of travel.  In other words, if site A and site B have a single route of medium resistance connecting them while site A and site C have five possible connecting routes of the same resistance, travel between sites A and C will be rated as easier than travel between sites A and B.  While distance is a factor, this may hold true even if site C is significantly further away than site B. 

The article’s tables and figures show an interesting comparison between the results of the two modelling strategies, but Howey concludes by emphasizing each tool has its strengths and weaknesses.  Circuit modelling is clearly useful in considering a larger portion of the landscape as potential routes of travel; humans are not perfect optimizers of their energies and surroundings, after all.  However, circuit modelling can overlook cases in which a single route is overwhelmingly preferred (such as water routes).  In the latter case, least cost path modelling would provide a more useful picture. 

The article may be a bit dry, I suppose, but I followed the suggestion to explore UWF’s library databases and went with a topic aligning with the archaeology track.  The article can be accessed at: http://dx.doi.org.ezproxy.lib.uwf.edu/10.1016/j.jas.2011.03.024. 

GIS 5103 Module 2: Geoprocessing in ArcGIS

This week continued our introduction to Python programming.  Unlike last week, however, this time we used Python script to utilize ArcGIS tools to produce output data.  The first step was to create a model using ModelBuilder to produce the layer pictured above.  This model needed to clip a layer representing a large area of soils to a small layer representing a basin, then to select from the soil layer all areas classified as "not prime farmland," and finally to remove from the basin layer areas that coincided with this soil selection.  The model was then exported as a Python script for further exploration and editing.

It was at this point that I experienced an issue.  Prior to exporting the model as a Python script, we were required to set our model variables as model parameters.  Because my model had five variables, the instructions' reference to four variables led me to not set the final variable as a model parameter.  I proceeded with the assignment, following the instructions to make the script operational outside of ArcMap by entering the file paths for the soils and basin layers and including the line of code to allow data overwriting.  However, I kept getting a "Parameters are not valid" error when I ran the script.  I eventually stumbled upon my simple mistake of not setting my final variable as a model parameter; the script ran smoothly afterward.

I enjoyed this introduction to using Python script for geoprocessing.  Even my little error described above taught me how important it is to pay attention to detail when programming.  I am looking forward to learning more in the next module.

GIS 5103 Module 1: Introducing Python

Thus begins our journey into the world of Python programming for GIS.  The screenshot above is the result of running a provided Python script to create the folders we will use for our assignments for the course.  In addition, we were initiated into Python programming by creating our "Hello world" script, a nearly-universal first line of code for the novice programmer.  And novice I surely am; the last formal instruction in any form of programming was with BASIC eons ago in an intro to computer science course.  Other than some fiddling with HTML here and there, this will be the first time I expect to be able to accomplish anything useful with coding.  I am looking forward to progressing through the course and learning more of Python.