Thanks for this post, it was very informative. The problem of what value to use in No Data areas is a challenging one, especially since different GIS systems behave differently in this regard. Your approach seems to make sense. I tested some lidar data downloaded from the Digital Coast Data Access Viewer in the MapWindow GIS system, which OpenNSPECT uses. MapWindow recognized the NoData value as "no data" and behaved appropriately.
I also tested the same data set with Esri ArcGIS and have a few observations that might be helpful to others.
In addition to using the color ramp technique to generate statistics for TIFFs, you can directly generate them using the "Calculate Statistics" function in Arc Catalog: just right click on the target raster in Arc Catalog pane.
The continued display of the NoData areas is odd and seems to be a function of the Stretched color ramps. If you use a classified color ramp, you can show the NoData areas with a different color, or not at all, as desired. This option appears in the Stretch GUI, but doesn't seem to work. Oddly, if you export the TIFF to a grid format, it still doesn't work, which makes me think it is a data value issue, not a file format issue.
If you really want those background NoData values to not show up (Which I do!), here is a trick: a Con command to pick the areas that are valid, then the rest will be set to Esri's NoData value::
Con( "Filename.tif" > - 40,"Filename.tif"),
where Filename.tif is my TIFF file. This works, regardless of the file format of the results, TIFF or Grid.
It is interesting that you can not just use:
SetNull(IsNull("Filename.tif"),"Filename.tif"), or SetNull(Con( "Filename.tif" < -40, "Filename.tif"), "Filename.tif")
The first of these essentially says, fill all the NoData places with NoData values. But it doesn't work. It appears that the NoData values in the TIFF are not recognized as NoData for the purposes of the IsNull command. So if you do this, you get the original data at all locations. The second of these says, for areas with values less than -40 (chosen since all the "real" data are above that), fill them with NoData values. This approach fails, but for a different reason: The Con command recognizes the current NoData value as NoData, therefore it doesn't perform any calculations for those areas. Therefore, nothing matches the SetNull criteria, so everything is set to the original data. Very odd.
However, you can trick the system by using a Con command to pass on values where you have legitimate data values. Where you don't have values that meet the Con criteria, Esri fills in Esri NoData values, which then work as desired.
All this indicates to me that Esri has at least two ways they evaluate NoData values: one approach is used in the Stretched color ramp, IsNull, (and other?) functions, and another one is used in the Classified color ramp and Con (and other?) functions. The value you provide via the DAV system for a NoData flag is treated differently in these, whereas Esri's NoData value is treated consistently. Interesting.
Thanks again, I look forward to hearing more about TIFFs. I use them a lot, but they do have some odd behaviors.
Dave Eslinger
March 4, 2013
Thanks for this post, it was very informative. The problem of what value to use in No Data areas is a challenging one, especially since different GIS systems behave differently in this regard. Your approach seems to make sense. I tested some lidar data downloaded from the Digital Coast Data Access Viewer in the MapWindow GIS system, which OpenNSPECT uses. MapWindow recognized the NoData value as "no data" and behaved appropriately.
I also tested the same data set with Esri ArcGIS and have a few observations that might be helpful to others.
Con( "Filename.tif" > - 40,"Filename.tif"),
where Filename.tif is my TIFF file. This works, regardless of the file format of the results, TIFF or Grid.
It is interesting that you can not just use:
SetNull(IsNull("Filename.tif"),"Filename.tif"), or SetNull(Con( "Filename.tif" < -40, "Filename.tif"), "Filename.tif")
The first of these essentially says, fill all the NoData places with NoData values. But it doesn't work. It appears that the NoData values in the TIFF are not recognized as NoData for the purposes of the IsNull command. So if you do this, you get the original data at all locations. The second of these says, for areas with values less than -40 (chosen since all the "real" data are above that), fill them with NoData values. This approach fails, but for a different reason: The Con command recognizes the current NoData value as NoData, therefore it doesn't perform any calculations for those areas. Therefore, nothing matches the SetNull criteria, so everything is set to the original data. Very odd.
However, you can trick the system by using a Con command to pass on values where you have legitimate data values. Where you don't have values that meet the Con criteria, Esri fills in Esri NoData values, which then work as desired.
All this indicates to me that Esri has at least two ways they evaluate NoData values: one approach is used in the Stretched color ramp, IsNull, (and other?) functions, and another one is used in the Classified color ramp and Con (and other?) functions. The value you provide via the DAV system for a NoData flag is treated differently in these, whereas Esri's NoData value is treated consistently. Interesting.
Thanks again, I look forward to hearing more about TIFFs. I use them a lot, but they do have some odd behaviors.