Hashing is the process of taking an input of variable length and mapping it to a fixed-size output value. These functions are often referred to as hashes. For instance, a basic hashing algorithm might reduce a text string to a single character based on its first letter, resulting in a hash like “w” for the string “we love labs.” However, this would be a poor hashing algorithm in practice due to potential collisions where different inputs produce the same hash.
Hashing applications
In computing, hashing algorithms are used in various ways. Hash maps (or hash tables) organize data using hashing algorithms, and these algorithms are also employed to generate checksums for error detection during data transmission or storage.
Cryptographic hash functions
In some cases, you might need to utilize an algorithm falling under the category of cryptographic hash functions. As the name implies, these are hashing algorithms specifically suited for cryptography. Such hash functions become necessary when generating digital signatures, securely storing passwords, or creating file fingerprints.
For instance, hashes are frequently employed to identify potentially malicious files. Since changing a file’s name is easy, attackers often distribute the same malware under various names. By generating a hash based on a file’s content, security experts can compare these hashes with other samples using services like VirusTotal, identifying different strains of malware regardless of the attacker’s chosen filename .
Hashing algorithms
In the section below we introduce some commonly used hashing algorithms with a high-level overview of where they are used.
MD5
Message Digest 5 (MD5) was designed by Ronald Rivest in 1991. Originally intended to be used as a cryptographic hash function, it has since been found to suffer from several vulnerabilities. It is now commonly used to generate checksums and verify data.
SHA-1
Secure Hash Algorithm 1 (SHA-1) is a cryptographic hash function which takes an input and produces a 40-character hexadecimal number. In recent years many organisations have moved away from using SHA-1 due to security concerns and the risk of collisions.
SHA-256
SHA-256 is one of six hashing functions which forms part of the Secure Hash Algorithm 2 (SHA-2) family. Designed by the National Security Agency (NSA) as a replacement for SHA-1, SHA-2 algorithms are used by a range of protocols including TLS, SSL and PGP.
CRC-16 and CRC-32
Cyclic Redundancy Check (CRC) is an error-detecting algorithm that is often used to identify errors in data which have occurred during storage and transmissions. CRC-32 is used by Ethernet and SATA compliant devices, whereas CRC-16 is used in a range of applications such as Bluetooth and Modbus protocol.
Hashing tools
There are several different ways to generate a hash. These have been broken down into command line tools and Graphical User Interface (GUI) tools. Although there are many other web-based tools available online that will produce the same results, they suffer from the added drawback of sending data to an unknown third party.
Command line
Windows, Linux, and macOS all provide a variety of command line tools for generating hashes.
- In Windows environments since Windows 7, you can use certutil.exe to generate file hashes. The basic syntax for this tool is:
CertUtil -hashfile <file path> [hash algorithm]
Alternatively, you can use the Get-FileHash cmdlet in PowerShell.
- In Linux operating systems, several commands exist for generating different types of file hashes.
md5sum, sha1sum, and sha256sum can be used to generate the corresponding hashes.
- On macOS systems, the commands vary slightly:
MD5 hash, use md5 <file path>.
SHA-1 hash, use shasum -a 1 <file path>
SHA-256 hash, use shasum -a 256 <file path>
Some examples in macOS:
Examples: hashing using command lines in macOS
