Identifying and addressing spatial problems between feature classes is essential for maintaining the quality of the Utility Network. Intersect Validation is a critical test in maintaining the spatial integrity of utility networks. This powerful test, available in GeoData Sentry, detects where features from one feature class fail to intersect with features from a second feature class.
Intersect errors can have far-reaching impacts, affecting feature-to-feature associations and potentially causing inaccuracies in data analysis and network tracing. Detecting and resolving these errors is vital to ensuring a smooth transition to the Utility Network.
It’s important to note that not all Intersect errors create Dirty Areas in the Utility Network. If the junction or device feature is far enough away from the edge it should be intersecting, a Dirty Area may not be created. This highlights the significance of the Intersect Validation test, as it provides an automated method to detect these errors that might otherwise remain unnoticed until data interaction occurs.
Another crucial step in maintaining data integrity within utility networks is Invalid Geometry Validation. This comprehensive test identifies features with invalid geometry, encompassing lines, points, polygons, and annotation. It detects thirteen different types of geometry problems and plays a key role in ensuring the accuracy and reliability of the data.
Among the most common issues detected by this test is NULL Geometry, highlighting the significance of addressing such problems promptly. Each feature identified by this test requires careful review and correction to rectify the geometry issues.
Each feature identified by the Intersect and Invalid Geometry tests requires correction to resolve the issues so full utilization of advanced Utility Network functions can be realized.
Need some help with your data validation? Reach out today – we’re happy to help!
Overshoot/Undershoot Validation is a critical test in ensuring the spatial integrity of utility networks. This powerful test detects features within a feature class that fail to connect with features of the same class or an optional second feature class. By identifying spatial problems within and between feature classes, this test plays a vital role in maintaining network connectivity.
Small gaps, undershoots, or overshoots between edges may seem insignificant, but they can lead to significant issues in network tracing and spatial analysis. For instance, a service lateral line feature class that doesn’t fully connect to the main feature class may hinder spatial analysis when tracing to customer meters for outage management.
While standard tracing methods can help detect some of these issues, visual inspection is often labor-intensive and may not reveal all undershoot/overshoot features, especially in back-feed situations.
Overlapping Edge Validation is an equally important aspect of maintaining data integrity of utility networks. This powerful test identifies instances of overlap between edges, both within a feature class and between feature classes. Detecting and resolving overlapping geometries is essential for data accuracy and network functionality.
While overlapping geometries may not always create Dirty Areas, they can lead to errors when creating subnetworks. Each feature identified by the Overlapping Edge test requires correction to resolve the overlapping issues so full utilization of advanced Utility Network functions can be realized. Addressing these overlaps proactively is vital to ensuring a smooth and error-free subnetwork creation process.
Need some help with your data validation? Reach out today – we’re happy to help!
Referential Integrity is an important aspect of data management. GeoData Sentry can be used to detect any discrepancies or mismatches between two tables based on their primary key/foreign key relationships.
The primary key and foreign key relationship ensures that all rows in one table are associated with corresponding rows in a second table through a common key item. By running referential integrity tests in both directions, organizations can identify and address any orphan records that may exist in either table.
The Utility Network relies on accurate and reliable data associations to function effectively. Any Referential Integrity errors in the source data can potentially impact features associated with the Utility Network, such as metersetting to meter relationships. Ensuring the integrity of these relationships becomes crucial for a successful migration process.
By thoroughly examining and addressing referential integrity errors, organizations can ensure the integrity of their data and facilitate a successful migration to the Utility Network. Maintaining a robust referential integrity framework is vital for preserving data consistency and supporting the optimal functioning of the Utility Network.
We had a great time at Esri IMGIS 2023! It was fun to catch up with several of our existing customers and meet some new people. And we’re already on the calendar for next year!
It was amazing to see the amount of interest in our Utility Network Data Assessment. These are a few of the recurring questions we answered:
What types of tests do I need to run to help me with a data assessment? GeoData Sentry offers a wide variety of tests for data assessment. These include Geometric Validation, Data Model Conformance Validation and Logical Connectivity Validation. Each test type detects a different type of issue that may cause problems migration into the Utility Network. Some tests (such as Duplicate Geometry and Duplicate Vertex tests) will detect errors that will always cause dirty areas in the Utility Network. Other test will identify errors, but they may not cause dirty areas—however they may still be significant. For instance, a domain test that finds an invalid manufacturer for a valve or transformer is not as significant as an invalid Pressure or KVA Rating. The greatest benefit of GeoData Sentry is that it can run a variety of test types on the entire geodatabase. This allows for a clear picture of the current data quality. It also allows for a prioritization of errors such as dirty area producing errors, and high consequence attribute errors. Once the errors are prioritized, let the data cleanup begin, and Sentry will be there to help you re-validate the data over time.
How long does it take to configure data assessment tests? GeoData Sentry test generation gives you the capability to create hundreds, if not thousands of tests very rapidly. Tests are generated based on information that is stored in the geodatabase, such as domains, subtypes, relationships, geometric network connectivity rules. and the feature classes themselves. Sentry can generate Geometric Validation, Data Model Conformance and Logical Connectivity test suites in minutes so you can get to the important steps of prioritizing errors and cleaning up the database.
How can I track my data assessment progress? Coming soon to GeoData Sentry, we’ll offer a report analysis tool to compare two test runs at different times. The comparison report clearly shows deltas between the two reports where corrected errors are highlighted in green, new errors are highlighted in red as well as the unchanged errors. This report shows change over time and is essential to determining database cleanup status, as well as is new errors are being propagated as old errors are being corrected. Sentry gives you the ability to get a clear understanding of the current state of the corrections, while helping prevent new errors from being created. Look for this functionality in our next release of GeoData Sentry.
How long does it take to run a basic data assessment test configuration? Over the years, we have optimized GeoData Sentry not only to run on the entire geodatabase, but to run as quickly as possible. There are some test types that will run in seconds or minutes, while others may take longer. The volume of data, the test type, and the type of computer hardware factor into the speed of the testing. One of our largest customers runs about 8,000 tests every weekend between Saturday morning and Sunday morning so they have new reporting ready every Monday as they look to prepare their data for migration to the utility network.
As always, if you have any questions or want more information, please don’t hesitate to reach out at any time!
Annotation Validation is an important step in maintaining the integrity of feature linked annotation within a geodatabase. This validation process focuses on testing the accuracy and consistency of annotation expressions. Using GeoData Sentry, a separate test is created for each annotation expression, evaluating it against the actual annotation text strings and reporting any differences.
When conducting Annotation Validation, detecting differences between the database values and the annotation can provide valuable insights. Such differences may indicate that the annotation has been manually edited, rather than the database being updated automatically as intended. Additionally, it could signal an invalid relationship link between the OBJECTID of the feature and the FEATUREID of the annotation.
Annotation Validation becomes particularly important if the Utility Network migration plan involves replacing annotation with dynamic labeling. Any differences detected during validation will need to be carefully reviewed and evaluated for necessary corrections to either the annotation itself or the database values that drive the annotation expressions.
By conducting comprehensive Annotation Validation, organizations can ensure the accuracy and consistency of feature linked annotation. This process supports the successful implementation of dynamic labeling in the Utility Network, ensuring clear and reliable annotation information.
Duplicate Geometry Validation is a critical step in maintaining data accuracy and integrity within utility networks. This powerful test identifies duplicate geometries for points, lines, and polygons, ensuring that data remains consistent and reliable.
In the context of geometric networks, it is worth noting that duplicate features may exist and still allow for tracing and analysis. However, stacked features, which should not occur in the Geometric Network, pose challenges within the Utility Network. If stacking features becomes necessary for the Utility Network, assigning a vertical Z value or horizontal XY offset becomes essential to clear any Dirty Areas that may arise.
By conducting thorough Duplicate Geometry Validation, utility companies can proactively address any potential duplication and stacked features. Ensuring data accuracy optimizes network performance and supports efficient utility operations and reliable service delivery.
Duplicate Vertex Validation is an equally important aspect of maintaining data integrity within geometric networks. This test focuses on detecting duplicated vertices for edges.
In the context of geometric networks, it’s essential to recognize that duplicate vertices may exist and still allow for tracing and analysis. However, each edge with a duplicated vertex will create a Dirty Area that requires attention and clearing.
All of the errors detected by the Duplicate Geometry and Duplicate Vertex test will create Dirty Areas in the Utility Network and must be addressed so full utilization of advanced Utility Network functions can be realized.
This is just one of many Spatial Relationship Tests that GeoData Sentry can run.
Want to try it with your own data? Reach out today – we’re happy to help!
Subtype Domain Validation plays a key role in maintaining the integrity of subtype-level coded and range domains within a geodatabase. This GeoData Sentry Test Suite focuses on verifying the validity of values within domain-controlled columns at the subtype level. By automatically generating the test suite from both subtype and domain rules defined in the geodatabase, organizations can effectively assess the accuracy of their data.
Similar to Domain Validation, Subtype Domain Validation ensures that values within domain-controlled columns are valid. This test suite includes a subtype filter, making it essentially a domain test with the added layer of subtype specificity.
In the context of the Utility Network, the aggregation of multiple subtypes into line, junction, and device feature classes is a critical aspect of migration. The assignment of domains by subtype becomes crucial for successful migration. Errors related to subtype-dependent domains are likely to have a significant impact on the migration process to the Utility Network.
By conducting thorough Subtype Domain Validation, organizations can verify the validity of values within domain-controlled columns at the subtype level. This ensures data integrity, streamlines migration efforts, and supports the successful implementation of the Utility Network.
Geometric Validation plays a vital role in ensuring the smooth migration from Geometric Network to Utility Network architectures. The two architectures handle geometry errors differently, making the detection and correction of these errors important for a smooth and successful transition.
In the Geometric Network architecture, tracing and analysis are possible with the existence of low-level geometry errors. The Utility Network architecture, most geometry errors create a Dirty Area. To establish Utility Network subnetworks, it becomes necessary to clear all Dirty Areas in the network topology. It is important to detect and correct these issues well before they become Dirty Areas in the Utility Network.
GeoData Sentry offers several tests that validate feature geometry. These tests can be run on your current production geodatabases to detect these errors and proactively make corrections prior to migration.
Duplicate Geometry Validation – detects duplicate geometries for junctions and edges.
Duplicate Vertex Validation – detects duplicated vertices for edges.
Overshoot/Undershoot Validation – detects small gaps, undershoots, or overshoots between edges.
Overlapping Edge Validation – detects overlap between edges, both within a feature class and between feature classes.
Intersect Validation – detects where junctions do not intersect with edges.
Invalid Geometry Validation – detects 13 different types of invalid geometry.
Disconnected Edge Detection – detects edges that do not connect to any other edge. These orphan edges will become untraceable features in the Utility Network.
Cutback Validation – detects where an edge cuts back at an acute angle creating jagged geometries.
Minimum Feature Length Validation – detects edges that fall below a user-defined feature length threshold that may present as nearly zero-length edges.
The significance of identifying and rectifying geometry errors cannot be emphasized enough. Each of these tests is designed to detect problems that create Dirty Areas, ultimately hampering subnetwork creation and restricting the full utilization of advanced Utility Network functions. By taking proactive steps to resolve errors, utilities can streamline the migration process and establish an accurate Utility Network right from the outset.
Need some help with your data validation? Learn more about Geodata Sentry, or reach out today—we’re happy to help!
Edge Edge Junction Validation is a crucial component in maintaining the integrity and accuracy of geometric networks. This advanced test identifies instances where two edges are incorrectly connected at a junction, violating the defined connectivity rules.
Geometric networks rely on specific rules to govern edge-to-edge connections at junctions, ensuring a cohesive and efficient utility system. For example, according to the connectivity rules, Steel Distribution Main should only connect to Steel Distribution Main at a Regulator, and Overhead Conductors should exclusively link to Underground Conductors at Riser junctions.
By conducting the Edge Edge Junction Filter Test, organizations can swiftly detect any deviations from these rules, guaranteeing that edge-to-edge connections at junctions align precisely with the defined geometric network guidelines.
Maintaining accurate geometric network connections is paramount for efficient utility operations, effective data analysis, and seamless network tracing. Leveraging Edge Edge Junction Validation allows utility companies to uphold data consistency, enhance network performance, and deliver reliable utility services.
Domain Validation plays an important role in maintaining the integrity of feature-level coded and range domains within a geodatabase. This GeoData Sentry Test Suite focuses on verifying the validity of values within columns that are domain controlled. By automatically generating the test suite from the domain rules defined in the geodatabase, organizations can effectively assess the accuracy of their data.
Domain errors detected during validation can have implications for migrating to the Utility Network. It is essential to ensure that all values within domain-controlled columns are valid to guarantee a smooth migration process. Specific values related to valve or switch status, active or abandoned flags, pressure or phasing, for example, can significantly impact the creation of subnetworks in the Utility Network.
While some domain errors may have little to no effect on migration, they could still impact symbology or definition queries in the ArcGIS Pro Project for the Utility Network. Therefore, it is crucial to evaluate all domain errors, considering their potential impacts on both the Utility Network and the ArcGIS Pro Project.
By conducting comprehensive Domain Validation, organizations can ensure the accuracy of their data values, mitigate migration challenges, and maintain the functionality of their geodatabase in the Utility Network.