Template File Format

WallsMap supports a specially-formatted text file, one with extension .tmpshp, that can serve two purposes. First, if it exists as one of the shapefile's components (i.e., with the same base name), it can control how attributes are initialized, edited, and displayed. You can use it, for example, to dynamically initialize fields with location-dependent data, to specify alternative field values to be presented in drop-down lists, and to define text that appears when the mouse pointer hovers over a field name.

Second, if it's selected as the template in an export operation, it can determine the attribute table structure of the exported shapefile. When it's used this way you can restructure a shapefile by changing the number and types of attribute fields. You can initialize fields with constants, or values that depend on location, such elevations and quadrangle names. You can also filter the records, selecting for output only the ones where one or more attribute fields have any of a specified set of values.

For the time being you'll need to use a text editor to do any customization. To start the process, perform an ordinary shapefile export (without specifying a named template file) while enabling the option "Force creation of template component." (See Exporting and Merging.) You might also check the "Initialize empty shapefile" box, since you likely won't need the actual shapefile output. This operation will create a .tmpshp file that reflects the existing structure, one that you can load into your text editor and change. Your edits must be compatible with the format described in this section.

When a shapefile has a template component, you can review or edit it by right-clicking the shapefile's line in the Layers window and selecting "Open <name>.tmpshp" from the context menu. Before automatically launching your default text editor, the program informs you that any changes you make won't take effect until the shapefile (or project) is closed and reopened.

Components of a Field Definition

The only lines in the file that are processed begin with a period, hence an attribute field definition would actually appear as shown in this generic description of a template directive:

(In these descriptions, optional parts are enclosed in square brackets. Since the sequence of directive arguments are important for their interpretation, double-quotes must occasionally be used to prevent separators like commas and spaces from changing the argument count. A default setting can be represented by "" if other arguments follow it.)

There can be a maximum of 255 field definitions, each identified by the .FLD keyword. This limit and the others described below are based on the dBase table format adopted by Esri, the originator of the shapefile format.

name is the field's name in the shapefile's DBF component. A legal name can be up to 10 characters long and have no embedded spaces. It should consist only of uppercase letters, digits, and underscores, with the first character being a letter. Some field names ending in an underscore (e.g., DATUM_) are given special treatment by the program.

type is a single character that indicates type of field, either N (number), F (float with exponent), L (logical: Y/N), D (date: yyyymmdd), C (fixed-length text), or M (variable-length text, or memo). Note that in general all field data is stored in the record as readable text, and the program won't reformat it for display purposes unless it's rich text in a memo field.

fld_length is the field's total length in characters. For N and F fields this number can have an appended period and count of displayed digits to the right of the decimal point. For example, specifying "12.3" would store the number -52.4 as "_____-52.400" in an N field and as "_-5.240e+001" in an F field (with "_" denoting a space). Although WallsMap uses exponential notation when filling F fields whenever possible, other programs may not. This shouldn't affect how the numbers are read and processed internally.

The maximum allowed fld_length and decimals components for N and F fields are 19.15 and 20.18, respectively. The maximum allowed length of type C fields is 254. Logical fields, date fields, and memo fields are forced to have lengths 1, 8, and 10, respectively. The record length, which is the sum of all field lengths plus one, cannot exceed 4000.

<< src_name specifies that the value for attribute <name> should be derived from a field named src_name in the source shapefile of an export operation. Without it, the field is considered new and will normally be blank filled. Note that field types C and M and types N and F are compatible in the sense that one can serve as the source for the other. In all cases, depending on the specified field lengths, truncation of text or loss of numeric precision is a possibility. src_name can also take the form described below under Creating a Record Number Field.

=(value1|value2|...) or =!(value1|value2|...) immediately following src_name specifies a set of one or more field values for use as a filter. In the first case, if any one of these values match the content of field src_name in an exported shapefile, the corresponding record is included in the export. The expression "=()" specifies that a non-blank field content will be sufficient for inclusion. In the second case (=!(...)), the record is included if the field content fails to match any of the values. Although a selective export can be accomplished from a table view, selection via a template can be convenient if it's an operation that's done often. Also, a commonly-used template can be be named "Export_shapefile name.tmpshp", such that whenever you select it by browsing a folder's content, the Output file box of the Export dialog is automatically filled with the desired output pathname -- that is, the template's pathname with "Export_" is removed.

?(..)  is an optional control expression beginning with "?(" and ending with ")" that can override the blank filling of new fields. It can also define special field attributes recognized by WallsMap during editing operations as long as this file, with extension .tmpshp, is kept alongside the shapefile's other components. Possibilities include displaying alternative choices in list boxes and retrieving location-dependent values (county names, elevations, etc.) from another layer in the project. The control expressions are described below under Controlling a Field's Behavior.

Note: The ?(..) control expression, when present, is the only part of a field definition that the program examines when the template is used as a shapefile component.

Creating a Record Number Field

Programs that upload shapefile data to GPS receivers (or convert shapefiles to GPX files) normally require that you supply the name of a text field to be used as a waypoint identifier or name. This field must store values unique to each record, and this uniqueness must be preserved when the text is truncated to the maximum length supported by the receiver model. (For a Garmin GPSmap 60CSx this limit is 14 characters.) The following definition, for example, will add such a field in an export operation:

.FLD LABEL  C 30  << |AK%03u.|NAME

The text inside the bars on either side of the number format specification, "%03u", and the source field name, "NAME", are all optional ingredients. In this example, if record number 19 has value "Cenote Agua Gudalupana" for field NAME in the source shapefile, then "AK019.Cenote Agua Gudalupana." will be stored in the LABEL field. If this record were uploaded to a Garmin 60CSx using the free program DNRGPS, the shortened waypoint identifier derived from LABEL would be "AK019.Cnt Ag G." A different program might simply truncate the text in LABEL, which is the case with Garmin's MapSource software.

Controlling a Field's Behavior

When a .tmpshp file is opened as a shapefile component, the only parts of FLD definitions that matter are the field names and the contents of optional ?(...) expressions. By contrast, when a .tmpshp file is selected for an export operation, the ?(...) expressions are processed only when they don't follow a source field name specification. They will, however, be preserved in the .tmpshp component that's created for the exported shapefile, thereby passing on any special field behavior. For field definitions without a <src_name> they can specify how the fields should be initialized, both during export and when the shapefile is later appended to. This feature is likely to be expanded with more options, but here's what's currently implemented:

Initializing fields with a constant value

Place the value, number or text, within a pair of vertical bars. In this example the source shapefile's record is ignored if the template is used in an export operation:

.FLD STATE  C  40  ?(|Texas|)

Location-dependent Initialization

If this is a point shapefile, you might want a field to be automatically filled with a value obtained by matching the point's location with features in another project layer. (This needn't be a visible layer). The following example causes the program, when initializing the COUNTY field, to search for a record in a polygon shapefile, Texas_Counties.shp, whose corresponding polygon contains the point. If one is found then that record's NAME field is used as the source; otherwise the field is initialized with Unknown. Note that when the template is a shapefile component the initialization occurs dynamically, both when a record is relocated and when a new record is created.

.FLD COUNTY   C 14   << COUNTY  ?(|Texas_Counties::NAME|Unknown|)

The control expression of this example, however, is ignored when the template is used in an export operation and not as a shapefile component. In the former situation the value for COUNTY is copied from the COUNTY field of the source shapefile. If you instead wanted to initialize a new field with the export (or to ignore existing field values), you would omit the "<< COUNTY" portion of the definition.

More than one polygon shapefile can be examined to retrieve an initialization value. For example, if used in an export template the following directive defines a new field with text values obtained from whichever shapefile record, if any, corresponds to a polygon enclosing the exported point. In this case, three geologic map shapefiles are examined in succession to determine a rock unit abbreviation. Since there is no trailing default value, say "|Unknown|", the field is left blank when no such polygon is found.

.FLD ROCK_CODE  C 10  ?(|EA_Geology::GEO_ABBREV|N_Bexar_Geology::GEOLOGY|GAT_250K::ROCKUNIT|)

It's also possible to initialize a field with an elevation if a digital elevation model (DEM) image of the right format is included in the project. Currently the only supported DEM image type is a specially-configured NTI file, one with an associated file component (extension .NTE) storing the actual elevations. In the example below, the control expression tells WallsMap to find the topmost DEM image whose extent includes the point's location, then retreive the corresponding elevation in feet. (It could be meters.)

.FLD ELEV_FT   N 5.0  << ELEV_FT  ?(|*.nti::feet|)

When location-dependent initialization is active outside of an export operation, any change of location (or record addition) will cause the affected fields to be reinitialized dynamically. The user can, however, make changes to those fields after the relocation provided they haven't been set read-only as described later in this topic. Currently dynamic initialization is only possible with fixed text (C) fields whereas memo fields can undergo location-based initialization during an export.

Combo box display

The control expression in the following field definition causes a drop-down list of alternatives to be shown whenever the user clicks the small button to the field's right:

.FLD ROCK_TYPE  C 30  << ROCK_TYPE ?(|Carbonate|Gypsum|Granite|Sediments|Volcanic|,1)

An optional integer, separated by a comma from the list of alternatives, specifies which one is to serve as the initial value in a new record. Here "Carbonate" is being specified as the default. If the integer is omitted, the field will initially be blank. In this example the user can also enter any text into the field without opening the list box. We can modify this behavior by ending the control expression with ",RO" to force the user to accept one of the alternatives. In that case the small button doesn't appear, the list box displaying only when the user clicks inside the field (or hits the Enter key).

Setting fields read-only

Any field where no special behavior is being specified can be made read-only by ending the definition with "?(RO)." Values in those fields will be displayed in gray. This can be useful for key fields whose values are program generated. A field's read-only status can be overridden during sessions where the program was invoked with the "-TS_OPT" command line option. For details, see Name for Signing Timestamps.

Important Note about Memo Fields

When type M (memo) fields are specified, a .DBT file containing the actual text will be created as one of the shapefile's set of components. An empty memo field requires just 10 bytes of space in the main table (DBF component) but a non-empty memo uses a multiple of 512 bytes in the DBT component.

Although WallsMap fully supports the display and editing of memo fields, as does mainstream database programs like Microsoft Access and OpenOffice.org, typical behavior for a GIS viewer is to tolerate their presence in the table while ignoring the DBT file. ESRI's products have exhibited a range of behaviors, with recent versions of ArcMap simply refusing to open shapefiles with memo fields.

Special Field Names Recognized by WallsMap

The following field subsets, whichever ones you specify, are automatically maintained by WallsMap assuming it has sufficient information. If you choose to use them, their sizes and types should be the same as those shown here. The values in these fields are always computed, never copied from a source. Nor can they be directly edited.

The two optional fields below are also handled automatically during record updates. They store a 19-char timestamp, a space separator, and an editor's name or initials. For an export operation you must specify a field length in the range 25 to 50. If the same timestamp field exists in the source shapefile, it is always copied unless it is explicitly initialized with either ?(|yyyy-mm-dd hh:mm:ss <name>|) or ?(||) following the field length. If it doesn't exist in the source, the field will be blank-filled unless you initialize it -- either explicitly or by entering a name in square brackets following the field length. The bracketed name will cause a timestamp to be generated at the time of export.

Unlike the location fields, the timestamp fields can be set to be directly editable during sessions where the program was invoked with the "-TS_OPT" command line option. The automatic updating of the UPDATED_ field is also suppressed when this setting is in effect. For more information, see Name for Signing Timestamps.

Optional Directives for Specifying Label Field, Search Field, and Key Field

After the field definitions, several types of additional field properties can optionally be assigned:

The .FLDLBL directive names a default field to be used for the record's label. Without this directive in a template, the first non-memo field is taken as the default label field when the shapefile is opened directly. If the user assigns a different field via the Symbols dialog, that assignment can only be saved in a project script. If the shapefile is opened as an existing layer during the loading of a project, the field specified in the project script (.NTL file) takes precedence over a template assignment.

The .FLDSRCH directive specifies a default set of fields to be examined in a text search operation. The default initial set consists of just the label field. Like the label field, the set of search fields is preserved in the project script.

This directive is required if "unlocated" records are to be recognized, or if the shapefile is to be used as the reference in a compare operation. (See Submitting a Shapefile Update and Processing a Shapefile Update.) It's best explained with an example: (Note the required use of double-quotes.)

.FLDKEY TSSID COUNTY 6 32.2 -103.3 "Texas_Counties::NAME,TSS_CODE"  Unknown UNK 10

Here, the unique key field is TSSID, which in general can be of C or N (integer) type. The second argument, COUNTY, specifies that the compare log will sort the mismatched record list by label field values within COUNTY field values. It will also identify each item by <sort field>: <label>. In this example, the COUNTY values are also used for the generation of new keys as described below. This requires that any shapefile selected for comparison with the reference have a COUNTY field as well as a TSSID field.

The next argument, "6", determines the precision used when comparing the latitudes and longitudes of point features. In this case, two locations would match if their latitudes and longitudes agreed to within 0.000001 (1/106) degrees. For backward compatibility, values of this argument outside the range 5 to 12 will be interpreted as 6.

Arguments, "32.2" and "-103.3", specify that features assigned geographic coordinates N 32.2, W 103.3 should be regarded by the compare function as having no location. Records having this status also won't be subject to location-based initialization. If no features are to be considered unlocated in this sense, specify "0" for both arguments.

The remaining arguments on this line relate to key generation. When creating an updated reference shapefile, the compare function can automatically generate keys for new records. Those keys can be either of two basic types, the first being a fixed-length string of characters followed by at least 4 digits filling the field. The TSS uses a 4-digit number prefixed with a 3-letter county code. As specified in this example, the function obtains the code by taking the record's COUNTY field value and searching for it in the NAME field of Texas_Counties.shp. The matched record in Texas_Counties will then have a field named TSS_CODE containing the prefix. Since the length of this field is 3 and the length of field TSSID is 7, it's determined that the key is a 4-digit number with a 3-character prefix. If the highest-numbered key for a feature in Bandera Co. is BAN0292, the next key assigned to a feature in that county will be BAN0293.

The directive's last three arguments are optional. If the COUNTY field contains "Unknown", the Texas_Counties shapefile is bypassed and "UNK", is taken as the key prefix. When the location of a new or relocated feature is outside the boundary of its designated county and at least 10 meters distant, a caution message is written to the compare log. This check of location occurs if Texas_Counties is a polygon shapefile, one that's likely also being used for location-dependent initialization of the COUNTY field.

The second kind of key that can be auto-generated is an n-digit numeric string with an unchanging m-character prefix, the length of the field being m+n. If field length is 7 and m is 2, "US00126" is a valid key. While m can be 0, n must be at least 4. For this type of key, the prefix is specified explicitly:

.FLDKEY ID STATE 6 0 0 0 "US"

To specify pure integer keys, enter a pair of double-quotes ("") as the last argument. If the key field is of type C, leading zeros instead of spaces will be inserted to fill the field. Finally, if there are no arguments following the coordinates, the compare function won't assign new keys.

This directive, of which there can be several, specifies a field to be ignored by default in a Compare with other layers operation. An optional second argument can specify a polygon shapefile layer (and field) to use for initializing <field name> when an updated reference is being created by the compare function. See Initialization of Ignored Fields under Processing a Shapefile Update.

Field Descriptions to Appear as Tooltips

Finally, the file can have lines starting with .FLDDESC, each having this form:

Whatever field descriptions you provide must appear after the field definitions, their order respect to each other being irrelevant. The program will display them as tooltips when the mouse pointer hovers over a field name. Note that the entire description, including keyword FLDDESC, must appear on one line. To force a line break within a tooltip, embed "\n" (without the quotes) within the single line of text.