Safeguarding Data Security and Customer Privacy with One-Way Cryptographic Hashing
At Trestle, protecting customer data is fundamental to our mission. We continually enhance our data privacy measures to uphold customer trust and comply with stringent privacy standards. In this blog, we detail our journey of implementing one-way cryptographic hashing for customer query data, highlighting significant improvements in both security and performance achieved through this advanced approach.
Understanding the Network in Trestle’s Ecosystem
Our “network” comprises customer query data stored when customers opt-in for data sharing. Allowing Trestle to utilize this data benefits customers through enhanced accuracy, identifying potential risk signals and improved response ranking, and simultaneously empowers Trestle to continuously refine and optimize our data-driven solutions.
Although Trestle already secures this data within a highly protected, encrypted database environment — accessible only through private IP addresses and secured via SSH keys — we continually strive to elevate our data protection standards. To further strengthen our commitment to privacy and ensure absolute confidentiality, we implemented one-way cryptographic hashing, rendering the stored data permanently unreadable, even internally.
Why Transition to One-Way Hashing?
Encryption alone did not fully satisfy our stringent privacy requirements. We aimed to eliminate even the theoretical possibility of unauthorized data access. To achieve this, we adopted one-way cryptographic hashing, specifically using the SHA2-512 algorithm.
Unlike encryption, hashing irreversibly converts data into a unique, fixed-length string. Once hashed, the original data is impossible to reconstruct — even by us. Therefore, hashed data remains completely unreadable, providing unmatched security against unauthorized access.
This approach ensures that even in the unlikely event of a database breach, the hashed data remains unintelligible, significantly enhancing customer privacy and maintaining the trust our customers place in us.
Our Journey Towards Implementing SHA2-512 Hashing
Planning: Choosing SHA2-512
Our team selected SHA2-512 due to its high security, resilience against collision attacks, and widespread industry acceptance. A key consideration during planning was ensuring that hashing did not introduce excessive latency, potentially impacting customer experience.
Execution: Normalization for Consistent Hashing
Hashing is highly sensitive, even slight variations in data representation (like capitalization or punctuation) drastically change the resulting hash. Thus, normalization became essential to ensure effective matching:
- Phone Numbers: Standardized with numeric format and country code prefixes.
- Emails: Converted to lowercase uniformly.
- Addresses: Processed through Trestle’s internal normalization service, standardizing various formats into a consistent, lowercase digest.
- Names: Limited to four words; each word hashed individually. Names exceeding four words use the first word and the last three words for hashing, allowing precise and consistent matches.
For example:
- Phone Numbers:
- Original: +11212121212
- SHA2-512 Hash: 1c20a03d0549b5235a98b07a6629903986bbb5c2b25817c6720dc3dd1d9e07b43ae437ad59189200ac31a1b61abed7e8954088e17245ddf8f74a887ee4749850
- Emails:
- Original: example@domain.com
- SHA2-512 Hash: 46efb31e9c20d158cd68e2f8fa36cc09945e97901cad9a1edb29b0762ee5ca5e370e292232686aa23ed609bf3cd95ad92635f2a187ea3eeb300e84124dd3cff8
- Addresses: An address can be written in multiple ways, yet both of these normalize to the same canonical form and yield an identical SHA2-512 hash:
- Raw Input A: 123 Main St., Apt #5, San Francisco, CA
- Raw Input B: 123 Main St Apt 5, San Francisco, CA
Normalization Process (Example):
- Remove all punctuation (commas, periods, “#”)
- Collapse multiple spaces into one
- Convert to lowercase
- Standardize common abbreviations (e.g., “St” → “st”)
Both Address A and Address B normalize to:
- 123 main st apt 5 san francisco ca
SHA2-512 Hash: 9cc9c108dae633b4e42caeb461fadac9a59e83e8fbf0eb2458f57d87d6cf8ce775801c6b427ab1dab766d79d880dae32dd3c9167e9d15b20c50fa6286cbd5b9b
Each of these hashed values is irreversible, ensuring customer data remains entirely confidential, even within Trestle.
Advanced Hashed Name Matching Logic
By matching each queried name against our hashed network data, we achieve unparalleled accuracy and power crucial risk-signal detection, tracking both query velocity and frequency on a per-identity basis. Normalizing and one-way hashing every variant (nicknames, alternate spellings) into a single irreversible identifier ensures that “Liz,” “Lizzy,” and “Elizabeth” all map to the same record. The result: faster lookups, zero dead-ends, and the confidence that every search surfaces the right information while simultaneously fueling our security analytics.
Name matching did, however, pose unique challenges during implementation. While exact matches were simple, synonym-based scenarios demanded extra care. To address this, we generate and hash every relevant synonym or variant – then compare those hashes against our stored entries. This sophisticated approach preserves privacy through one-way hashes while maintaining the precision and reliability of both our response ranking and our fraud-prevention workflows. By carefully implementing hashed synonym matching, we successfully maintain precise, privacy-preserving name matching capabilities, enabling us to continually enhance the quality of outcomes delivered to our customers.
Optimizations for Performance and Latency
Introducing SHA2-512 hashing inevitably added latency (approximately 9 milliseconds at the 99th percentile) to our existing data processing pipeline. Given our commitment to high performance, this additional overhead required targeted solutions to offset the impact.
Because reducing the hashing time itself beyond this threshold was not practically feasible, our team focused on optimizing other components of our data retrieval and storage processes. This led us to critically evaluate our database indexing strategy to compensate effectively for the additional latency introduced by hashing.
Efficient Hash Indexing
We transitioned from traditional B-tree indexes to hash indexes specifically tailored for our hashed data:
- Hash Index Benefits:
- O(1) look-ups: Constant-time searches, even with billions of records.
- Optimized storage: Hash indexes use significantly less storage compared to B-tree indexes for large keys, such as 128-character SHA2-512 hashes.
For timestamp-related queries, we adopted BRIN indexes (Block Range Indexes) instead of traditional B-tree indexes:
- BRIN Index Benefits:
- Minimal Storage Overhead: BRIN indexes significantly reduce storage usage.
- Optimized for Range Queries: Ideal for sequential data like timestamps, substantially improving query performance compared to standard B-tree indexing.
Database Transaction Isolation Level Adjustment
Another pivotal optimization was changing our transaction isolation level from “Read Committed” to “Read Uncommitted.” This adjustment notably decreased database contention by reducing locks during concurrent read/write operations, effectively halving latency in many scenarios.
Final Achievement: Latency Improvement
Through meticulous implementation and optimization, we achieved a remarkable milestone:
- Hashing originally anticipated to increase latency; instead, our optimizations resulted in an overall latency reduction of approximately 50%.
Storage Impact and Manageability
While hashing inevitably increased data storage (from approximately 200 GB to 300 GB, a 50% growth), the benefits significantly outweigh the manageable increase in storage requirements. Achieving enhanced data security, unparalleled customer privacy, and significantly reduced latency justified this trade-off entirely.
Conclusion: Trestle’s Commitment to Security and Performance Excellence
Here is how the network data in our database looks after implementing SHA2-512 hashing:
Introducing one-way cryptographic hashing demonstrates Trestle’s unwavering commitment to advanced data security and superior privacy standards. By hashing sensitive data irreversibly, we ensure unmatched protection against potential breaches while preserving our ability to deliver consistently accurate and improved results.
This project highlights our proactive approach, consistently prioritizing security, privacy, and performance. At Trestle, innovation in data protection remains at the forefront of our mission, ensuring our customers always benefit from the highest standards of data integrity and security.
