GeoServer GetLegendGraphic Fails With “Width (0) and height (0) cannot be <= 0″

I had a weird issue with WMS GetLegendGraphic requests failing with the following message:

<?xml version="1.0" encoding="UTF-8" standalone="no"?><!DOCTYPE ServiceExceptionReport SYSTEM "http://michael.mapfeeder.net:80/geoserver/schemas/wms/1.1.1/WMS_exception_1_1_1.dtd"&gt ;<ServiceExceptionReport version="1.1.1" &gt ;<ServiceException>
java.lang.IllegalArgumentException: Width (0) and height (0) cannot be <= 0
Width (0) and height (0) cannot be &amp;= 0
</ServiceException></ServiceExceptionReport>

This was weird because my request definitely included non-zero height and widths, like so:

/geoserver/wms?LAYER=parcel_area&REQUEST=GetLegendGraphic&FORMAT=image%2Fpng&STYLE=michael_test&WIDTH=20&HEIGHT=20

Eventually I tracked it down to an issue with the SLD xml. The XML was valid, but GeoServer didn’t like the use of CDATA tags inside of <Name>,<Title> and <ogc:Literal> tags.

So anywhere with formatting like this

<NamedLayer>
<Name>
<![CDATA[Parcel Area]]>
</Name>
<UserStyle>
<FeatureTypeStyle>
<Rule>
<Name>
<![CDATA[ Low Density Residential ]]>
</Name>
...

had to be changed to

<NamedLayer>
<Name>Parcel Area</Name>
<UserStyle>
<FeatureTypeStyle>
<Rule>
<Name>Low Density Residential</Name>
...

Once I got rid of the CDATA tags everything worked beautifully.

This issue was on GeoServer 2.5. YMMV, etc.

Posted in Computers, GIS Portfolio, Programming | Tagged , , , , | Leave a comment

Running ArcPy Scripts on a Linux Cluster or Super Computer

Sometimes you need more processing power from ArcPy than your ArcMap or ArcServer install can give you. In such cases your friendly campus super-computer may be helpful.

Your super-computing administrators can help you with the usual scripting, job allocation and other issues that come up with super computer jobs, but they may not be as familiar with the quirks of getting ArcPy on Linux working on the compute nodes of their super-computer.

Use Case

We used a 1 TB raster mosaic DEM of the state of Minnesota as input and ran the Solar Radiation Analysis tool on the whole state. It required more computing power than we could get on the servers we had available and so we processed some of it on the UMN Super Computer, Itasca.

What You’ll Need

  • ArcPy for Linux along with appropriate licenses
  • A network drive shared with all compute nodes on your cluster

We installed ArcPy for Linux by installing ArcGIS Server for Linux. ArcGIS Engine is also available for Linux and comes with ArcPy. These instructions will be for Arc Server, but I suspect that ArcGIS Engine are likely similar.

Background

ArcPy on Linux actually runs inside of [Wine](http://www.winehq.org/). Wine allows Windows applications to run on Linux. When you run ArcGIS Server for Linux’s install of Python you are actually running the same Windows version of python that you would be in ArcMap or ArcGIS Server for Windows.

We need to make a few modifications so that it will work in a clustered environment.

The main problem is that Wine expects that each user will have their own .wine directory. Wine implements the Windows registry. If you have multiple users (or nodes in the cluster) using the same .wine directory the registry will get corrupted when multiple users try to write at the same time.

Installation Instructions

Allow DOTWINE Override

Install ArcServer to a directory which is available to the compute nodes. Make sure that the directory is available through the same file path on the node you’re working on and the compute nodes.

Edit the arcenv file located in arcgis/server/framework/etc/arcenv. We need to make it possible to override the DOTWINE variable which points at the user’s windows applications (including the Windows version of Python and the Python libraries).

Replace the line that says:

DOTWINE=”$ARCGISHOME/framework/runtime/.wine”

with

if [ "x$DOTWINE"="x"];then
DOTWINE=”$ARCGISHOME/framework/runtime/.wine”
fi

Add Xvfb Support

ArcPy on Linux requires Xvfb. The call to Xvfb is made in init_Xvfb.sh. ArcGIS Server installs Xvfb libraries for RHEL and SuSE, and looks for the Ubuntu Xvfb libraries. The cluster we used ran Debian but was detected as Ubuntu.

We symlinked the RHEL Xfvb libraries to a directory named Ubuntu and everything worked fine.

cd server/arcgis/server/framework/runtime/xvfb
ln -s RHEL/ Ubuntu

In our case the compute nodes were missing libXdmcp.so which Xfvb needs, but the login node had them. Copy them from the login node to the compute nodes:

cd server
mkdir -p arc/libs
cp -L /usr/lib64/libXdmcp.so* arc/libs

Scripting Setup

With Xvfb set up and the ability to override the default .wine directory, it’s time to create a setup script. This script is what will be submitted to the server. It will set up environment variables and then launch the python script you actually want to run.

We start by setting up some variables and then make a fake ArcGIS server install. The fake install is a bunch of symlinks back to the real install — for most files and directories. The .wine directory however is actually copied so that we’re working off of our own copy.

The symlinks are needed because inside the .wine directory are relative symlinks which point to the ArcGIS Server files. You can’t just move the .wine directory and expect it to work.

  
#!/bin/bash


# Make a symlink clone of a directory
# If the destination directory exists, but is a symlink, the symlink 
# will be removed and replaced with an actual directory 
# actual directories will not be replaced
function fakedirs {
    sourcedr=$1
    destdir=$2

    echo "Making fake $destdir from  $sourcedr"

    # Make the dest dir if it doesn't exist
    if [[ -h $destdir ]];then
        rm -f $destdir
    fi
    mkdir -p $destdir

    # Switch to the dest dir
    pushd $destdir >/dev/null

    # For every file in the sourcedir, symlink to it in the dest dir
    for i in $sourcedr/*;do
        if [[ ! -e $(basename $i) ]];then
            ln -s $i
        fi
    done

    # Change back to the directory we were in
    popd >/dev/null
}


# Define the path to the ArcGIS Server install
REALARCINSTALL="/home/username/shared/arc/server/arcgis/server"

# Add the ArcGIS Server tools to your path
export PATH=$REALARCINSTALL/tools/:$PATH

# Add the arc/libs directory to your LD_LIBRARY_PATH so that libXdmcp.so* get picked up
export LD_LIBRARY_PATH=$REALDATADIR/arc/libs:$LD_LIBRARY_PATH

# Optionally set your wine debug level
export WINEDEBUG="-all"

# Define which directory you want to use for your wine install
# Note: This doesn't have to exist yet. 
# Note: This should be distinct for each job on the cluster. It 
# could be on the processing node's local hard drive, or it could be 
# on a network share, but it needs to be one .wine directory per job.
# In a PBS environment the variable $PBS_VNODENUM will ensure that 
# each job on the node gets its own number

FAKEWINETOP="/tmp/username/fakeWine_$(hostname)_$PBS_VNODENUM"

echo "Cloning Wine"
fakedirs $REALARCINSTALL $FAKEWINETOP
fakedirs $REALARCINSTALL/bin $FAKEWINETOP/bin
fakedirs $REALARCINSTALL/framework $FAKEWINETOP/framework
fakedirs $REALARCINSTALL/framework/runtime $FAKEWINETOP/framework/runtime
    
# Copy the actual wine directory now
# If there's already an actual .wine directory remove it and replace it
echo "Copying actual .wine dir #1"
if [[ -h $FAKEWINETOP/framework/runtime/.wine ]];then rm .wine;fi
rsync -rl $REALARCINSTALL/framework/runtime/.wine/ $FAKEWINETOP/framework/runtime/.wine/

echo "Copying actual .wine dir #2"
# If there's already an actual .wine directory remove it and replace it
if [[ -h $FAKEWINETOP/bin/.wine ]];then rm .wine; fi
rsync -rl $REALARCINSTALL/bin/.wine/ $FAKEWINETOP/bin/.wine/


#######################################################################################
#
# Copy any data you need locally for this job instance here
#

#######################################################################################
#
# Finally, we launch the actual job. Since Wine is still a little 
# crashy, we're going to try to start the job up to 100 times
#

# After crashing we sleep for a random time, up to 19 seconds. If 
# we're running multiple jobs on the same physical node we can have multiple
# jobs trying to start Xvfb at the same time which seems to cause 
# intermittent race conditions. The random sleep time worked around this.

MAXTRIES=100
echo "Launching script on $(hostname)"
while [[ $MAXTRIES -gt 0 ]]; do 
    $REALARCINSTALL/tools/python /home/user/path/to/your_code/myScript.py 2>&1
    echo "$(hostname) - wine down again ($MAXTRIES tries left) -"
    sleep $(( ( RANDOM % 10 )  + 10 ))
    ((MAXTRIES--))
done

echo "$(hostname) I tried to start python/wine 100 times. I'm giving up and going home."

Other Considerations

Besides getting Wine/Python working there were several other issues we came across.

Data transfer

Our input data was a one terabyte raster mosaic. For small jobs (20-40 nodes) the nodes were able to access the raster mosaic concurrently over a network drive. Beyond this scale delays were noticable. Since we were only reading from the raster mosaic I suspect that the issue was the network share, and trying to access the same set of files from that share from 200+ clients.

To work around this issue, we used the same symlinking trick we did for the ArcGIS server install to duplicate the structure of the file geodatabase which held the raster mosaic. The referenced rasters were symlinked to a path which corresponded to the location that the GeoDatabase expected to find them. The specific tiles needed for the computations were copied to the local compute node.

Incremental writes

In our case our output files were 500-700 MB each. The Solar Radiation Analysis tool writes these output files incrementally. Unfortunately, the network file system we were on did not deal well with multiple (100s or 1000s) of concurrent incremental large file writes.

For large jobs we experienced lots of data corruption. We found that the best solution was to write the resulting file to the local compute node’s disk and then copy the file all at once. Using this method avoided the pattern of corruption we were experiencing.

Posted in Computers, GIS Portfolio | Tagged , , , , , , , , | Leave a comment

2014 Boundary Waters Trip Report

Well, I finally did it. I’ve been wanting to take a Boundary Waters trip since 2011 when I went along with a scout troop. That trip was fun, but I wanted to go on a trip without all the teenage goofiness.

This year I was determined. I was going to go, even if it was a solo trip. Fortunately I ended up getting to go with some good friends. My wife (Caroline) and 8 year old son (Ryan) came, my brother-in-law Spencer, a friend from Church (Ricky), and some friends from school (Jie and Yuanyuan). So there were 7 of us all together.

Preparation

Our planned trip was June 9 – 14 (6 days) We ended up coming back a day early due to weather and gear malfunctions. More on that later!

Our actual route: Enter at entry point 24, Fall Lake. Went up Pipestone, up Basswood lake, down Basswood river, up Horse River, across Horse Lake, across Tin Can Mike lake, Sandpit lake and exited at Mudro.

We had planned on portaging from Sandpit back to Jackfish and then down to Fall Lake where we put in, but the aforementioned issues led us to alter our route.

bwca_trip

We self-outfitted, including planning all our own meals. You can see our meal list here.

We made a whole lot of jerkey. 3 large rump roasts worth. The butcher sliced it thin for me so I didn’t even have to do that part. We made maple, BBQ, lemon-pepper and spicy. They all turned out really well, but BBQ was the favorite.

DSCN0017

We borrowed 3 canoes from friends, and I made home-made canoe racks from scrap lumber to carry them. This one fits two canoes and is wrapped in an old yoga mat that someone left in my garage for the last year. If that someone needs it replaced, let me know. The other rack was similar, but fit just one canoe on our van.

DSCN0105

Here’s all our food. We ate very very well. Each person was responsible for carrying their own snack bag which they could eat whenever they wanted. The meals were divided up between people’s packs.

DSCN0128

I made my annual fishing gear purchasing trip right before we left.

DSCN0131

We treated our clothes with permethrin to keep ticks and mosquitoes at bay. We didn’t see a single tick, and I only got two mosquito bites so I think it probably worked.

DSCN0132

Off We Go!

Day 1

Here’s Ryan, Caroline an I about to start on our trip! Ryan is 8. I told him he could come with if he learned to swim, which he did.

DSCN0134

On our first day we traveled about 12 miles. We had two short portages and lots of paddling. We went across Fall Lake, up to Pipestone bay and then up to a camp site on Basswood lake. We got into Basswood at about 4pm and the site we were hoping for was taken. We had a slightly discouraging time finding a site in the rain but ended up getting a place to set up camp. A very very muddy place. Still, we kept warm, ate steak, got some sleep and made plans for the next day.

Day 2

Day 2 started with a very short canoe ride followed by our longest portage. It is marked as 361 rods, which is 1.1 miles. We double-portaged, which means we took 2 trips. So we walked 3.3 miles (2.2 with gear, 1.1 going back).

Between that portage, two smaller portages and a bunch of paddling we only traveled 6 miles total.

Here we are resting and planning after the long portage. We decided to sacrifice a rest day (we had planned on 2 rest days) in order to have two shorter travel days.

DSCN0142

After paddling downstream for a bit we got to an island with rapids on either side of it. The larger rapids is named wheelbarrow falls. I’m not sure that the smaller rapids has a name. We portaged the smaller rapids because the larger rapids portage is in Canada and we weren’t sure how that would work with our Chinese friends’ visas.

This portage was lots of fun. Ankle to knee deep water for much of the trail. Here’s Caroline getting ready to carry her stuff down the portage.

DSCN0146

Here’s a video of me carrying the canoe along the portage.

We found a nice campsite right before the mouth of the Horse river. There was a nice big rock which we fished and swam off of.

DSCN0151

Ryan caught a clam. It actually had his rubber worm inside it. He also caught a pike but it flipped off at the last minute and bounced down the rock back into the river.

DSCN0152

Ryan with his favorite rubber fish. It’s a good lure.

DSCN0154

Spencer had less luck. Sorry Spencer.

DSCN0161
Here’s Ryan jumping into the river.

 Day 3

Jie and Yuanyuan paddling by some rapids on the Horse River.

DSCN0170

There were 3 small portages around rapids, and three smaller rapids without portages. People going downstream could just ride through these smaller rapids, and we tried hard to paddle up them. Two of our three canoes made it up the first two rapids, One canoe needed a push to get up them.

Since the water was warm I just jumped in and pushed them.

On the third smaller rapid, no one could make it up by paddle power. We used a rope and I got upstream to pull people up it while they steered and paddled.

DSCN0177

The campsite our friend Andy had told us about on Horse Lake was taken, but we found a suitable replacement site near by. There was a large camping area of grass and dandelions where we set up camp.  The trail up to the latrines was a field of wild roses and goldenrod all in bloom. It was the prettiest walk to the toilet I’ve ever seen. There was also an abundance of reasonably burnable wood.

Here’s Ricky eating some trail mix by the canoe landing.

DSCN0183

Caroline and I hanging out at the camp site.

DSCN0184

We got some light rains so we set up the tarp. The rope it was hung over sagged, so we lashed two paddles together to make it taller. I mostly got the lashings right, but here Spencer is re-teaching me some of the finer points I had forgotten.

DSCN0186

We managed to get some fishing in before it got too late. Here’s my 3lb, 19 inch walleye and Ryan’s nice little bass.

DSCN0189

We made tempura battered walleye and bass and it was delicious. We later went back and caught some sunfish and more bass so we could use up the rest of the batter.

DSCN0190

Here’s Ricky with his little bass.

DSCN0193

And Jie with the first fish he ever caught! Nice work Jie!

DSCN0194

That night it rained pretty well and both our tent and Ricky’s tent started leaking. We moved the tents under the tarp and made it through the night just fine.

Day 4

Day 4 was a rest day. Sort of. It started out that way!

Caroline was washing Ryan’s face with a wet-wipe and bumped his loose tooth. It finally fell out into his grubby little hands. Nice work Ryan!

DSCN0197

Ryan spent some time “peeling sticks”. He likes to use his pocket knife to peel the bark off in different patterns.

DSCN0201

Ryan took some time to teach Yuanyuan how to peel sticks with a knife.

DSCN0202

By this point Caroline’s hair was looking pretty awesome. Does it remind you of a certain evil someone?

DSCN0207

Ryan caught a nice big sunfish. Spencer made up for his previous poor fishing record by catching 4 fish in 4 casts right before Ryan caught this one. After those 4 casts the fishing dried up again.

DSCN0212

Caroline and Yuanyuan share some hot chocolate.

DSCN0229

 

About 3 or so in the afternoon several of us were out on the lake fishing. Very quickly the weather turned and it started to rain. After about 15 minutes of light drizzle the wind picked up. It was a bit hard to get back to camp. Fortunately we were all in or near the bay and not out on the main part of the lake.

A troop of scouts out on Basswood actually capsized and had to be flown out.

The wind was gusty, 10-20 mph with gusts up to 30 mpg — at least that’s what the weather forecast said. About this time our tarp collapsed from the wind and rain and crushed our tent, snapping the poles. The tent was a Walmart tent Caroline and I bought 10 years ago, so it’s not really a surprise that it broke, just bad timing.

Between our broken poles, the wind and Ricky’s leaky tent we decided to relocate from the open grassy area down to the side of a little hill. We set up a sort of tarp lean-to and moved all the gear inside.

Little did we know that our tarp (about 5 years old, also from Walmart) was leaky too. The rain was so heavy that the little leaks added up quickly to puddles. Our sleeping bags got soaked and the temperatures dropped. We had Ryan sleep in with Jie and Yuanyuan who were using our friend Andy’s good tent. Ryan had a GREAT time playing telephone and other games with them. If they hadn’t been there Ryan would’ve been a sad miserable little fellow hanging out with us in our wet cold situation.

The rest of us did our best to stay warm and dry. Spencer managed to find a dryish spot within the tarp. Ricky set up his tent once the rain stopped around midnight and snuggled a water-bottle filled with near boiling water.

Caroline and I turned a canoe sideways and made a lean-to with some other tarps and our tent’s rain fly. I stayed up most of the night trying to get my clothes to dry and keeping the fire going.

It was easily my least comfortable night camping ever. I don’t know what the actual temperatures were, but the forecast said it was going to be in the high 30s to low 40s. With wet gear it was just plain uncomfortable and bordering dangerous. The fire didn’t project much heat since our wood was all damp and a little punky. What ended up working OK was to heat our gallon camp pot up to boiling temps and then snuggle it.

I managed to burn holes in my wool socks, 1 t-shirt and 1-long sleeve shirt, as well as adding small burning ash holes to my rain gear as I tried to get things dry.

leanto

Day 5

The weather forecasts on the morning of day 5 said that after midnight there would be more storms and rain. At this point almost everyone’s clothes were all wet, we were down a tent and our tarp was leaky. We decided to call it a trip and cut things short a day. The weather was beautiful the whole time we were traveling and it only started to get cloudy after we were on the road back home.

The closest entry point was Mudro, which is not where our vehicles were at. We decided to get to Mudro and try to bum a ride. Fortunately it worked.

Here’s Caroline with a full load of gear on a portage.

DSCN0246

 

Tin Can Mike lake was the first lake after Horse. It was absolutely beautiful with all sorts of rocky formations. I think I’d like to make that my next destination.

The portage from Tin Can Mike to Sandpit lake had this little waterfall. I tried to get Ryan to drink from it, but he was unimpressed. “NO! THERE’S DIRT”.

 

 

 

 

 

DSCN0252

We squeezed in a bit of last minute fishing along the way. Nothing was biting though.

DSCN0258

We made it to Mudro and I got a ride back to Fall Lake to pick up a vehicle. After some shuttling around we finally got back to Ely by about 4, ate a nice dinner at Subway and headed back to Minneapolis. We arrived safely home at about 11 PM and slept very very well.

Final Advice From Ryan

Finally, Ryan has some advice for all of you who are considering a trip to the Boundary Waters.

Posted in Something Interesting | Tagged , , , , , | Leave a comment