Top Related Projects
Project Wycheproof tests crypto libraries against known attacks.
Guidelines for low-level cryptography software
A curated list of cryptography resources and links.
Cryptomator for Windows, macOS, and Linux: Secure client-side encryption for your cloud storage, ensuring privacy and control over your data.
Quick Overview
The C2SP/wycheproof repository is a collection of test cases for cryptographic libraries, designed to help developers ensure the security and correctness of their implementations. It is named after the Wycheproof Mountains, a small mountain range in California known for its challenging terrain, which serves as a metaphor for the complexities and pitfalls of cryptographic programming.
Pros
- Comprehensive Test Suite: The repository contains a wide range of test cases covering various cryptographic primitives, algorithms, and use cases, helping to ensure thorough testing of cryptographic implementations.
- Vendor-Neutral: The test cases are designed to be vendor-neutral, making them applicable to a wide range of cryptographic libraries and implementations.
- Actively Maintained: The project is actively maintained, with regular updates and additions to the test suite, ensuring that it remains relevant and up-to-date.
- Open-Source: The project is open-source, allowing developers to contribute, report issues, and collaborate on improving the test suite.
Cons
- Complexity: The test suite can be complex and challenging to understand, especially for developers who are new to cryptographic programming or testing.
- Limited Language Support: While the test suite supports several programming languages, it may not cover all the languages and libraries used by developers.
- Potential False Positives: As with any test suite, there is a possibility of false positives, where the test cases may flag issues that are not actually security vulnerabilities.
- Dependency on External Libraries: The test suite may depend on external libraries or tools, which can introduce additional complexity and potential points of failure.
Code Examples
N/A (This is not a code library)
Getting Started
N/A (This is not a code library)
Competitor Comparisons
Project Wycheproof tests crypto libraries against known attacks.
Pros of wycheproof
- Comprehensive cryptographic test suite
- Regularly updated with new test vectors
- Supports multiple programming languages
Cons of wycheproof
- May require additional setup for specific language environments
- Learning curve for understanding test vector formats
- Limited documentation for some less common cryptographic algorithms
Code comparison
wycheproof:
public class EcdsaTest {
@Test
public void testVerify() throws Exception {
TestVectors vectors = TestVectors.fromJsonFile("ecdsa_test.json");
for (TestGroup group : vectors.getTestGroups()) {
// Test implementation
}
}
}
As the comparison is between the same repository (C2SP/wycheproof), there is no distinct code to compare. The repository contains test vectors and implementations for various cryptographic algorithms, but the structure remains consistent throughout.
Summary
Wycheproof is a project focused on providing cryptographic test vectors for various algorithms and implementations. It aims to improve the security of cryptographic software by identifying common implementation errors. The repository contains a wide range of test cases covering different cryptographic primitives and protocols.
The project's strengths lie in its comprehensive coverage and regular updates. However, users may face challenges in setting up the environment for specific programming languages and understanding the test vector formats. Despite these minor drawbacks, Wycheproof remains a valuable resource for cryptographic testing and validation.
Guidelines for low-level cryptography software
Pros of Cryptocoding
- Cryptocoding provides a comprehensive set of guidelines and best practices for secure cryptographic coding, covering a wide range of topics such as random number generation, side-channel attacks, and more.
- The repository includes a large number of code examples and test cases, which can be valuable for developers looking to implement secure cryptographic solutions.
- The project is actively maintained and has a strong community of contributors, ensuring that the guidelines and recommendations stay up-to-date with the latest developments in the field.
Cons of Cryptocoding
- The repository can be overwhelming for developers who are new to cryptography, as it covers a large number of complex topics and concepts.
- The focus on comprehensive coverage may come at the expense of depth, and some developers may prefer a more targeted approach to specific cryptographic problems.
- The repository does not provide a clear roadmap or development plan, which can make it difficult for contributors to understand the project's long-term goals and priorities.
Code Comparison
Cryptocoding:
/* Avoid using rand() for security-critical applications */
void get_random_bytes(void *buf, size_t len) {
int fd = open("/dev/urandom", O_RDONLY);
if (fd == -1) {
/* Handle error */
}
if (read(fd, buf, len) != len) {
/* Handle error */
}
close(fd);
}
Wycheproof:
public static byte[] generateRandomBytes(int numBytes) {
byte[] bytes = new byte[numBytes];
SecureRandom secureRandom = new SecureRandom();
secureRandom.nextBytes(bytes);
return bytes;
}
Both code snippets demonstrate secure random number generation, but the Cryptocoding example uses the /dev/urandom device directly, while the Wycheproof example uses the SecureRandom class provided by the Java standard library.
A curated list of cryptography resources and links.
Pros of awesome-cryptography
- Comprehensive resource collection covering various cryptography topics
- Regularly updated with community contributions
- Includes links to tools, libraries, and educational materials
Cons of awesome-cryptography
- Lacks practical testing capabilities for cryptographic implementations
- Does not provide direct code examples or test vectors
- May require additional research to determine the best tools for specific use cases
Code comparison
While a direct code comparison is not relevant for these repositories, here's a brief example of how they differ in content:
awesome-cryptography (README.md):
## Encryption
### AES
- [OpenSSL](https://www.openssl.org/) - TLS/SSL and crypto library
- [cryptography](https://cryptography.io/en/latest/) - Python cryptography library
Wycheproof (README.md):
## Test vectors
The test vectors in this project are available in JSON format:
- AES-CBC
- AES-GCM
- RSA-OAEP
Wycheproof focuses on providing test vectors and practical tests for cryptographic implementations, while awesome-cryptography serves as a curated list of resources and tools related to cryptography.
Cryptomator for Windows, macOS, and Linux: Secure client-side encryption for your cloud storage, ensuring privacy and control over your data.
Pros of Cryptomator
- User-friendly encryption solution for cloud storage
- Cross-platform compatibility (Windows, macOS, Linux, iOS, Android)
- Active development and regular updates
Cons of Cryptomator
- Focused on file encryption, not a comprehensive testing suite
- Limited to specific use case (cloud storage encryption)
- May not cover all cryptographic edge cases
Code Comparison
Cryptomator (Java):
public class AesGcm {
public static byte[] encrypt(SecretKey key, byte[] associatedData, byte[] plaintext) throws CryptoException {
Objects.requireNonNull(key);
Objects.requireNonNull(plaintext);
try {
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
Wycheproof (Java):
public class AesGcmTest extends TestCase {
public void testVectors() throws Exception {
JsonObject test = JsonUtil.getTestVectors("aes_gcm_test.json");
int errors = 0;
int numTests = test.get("numberOfTests").getAsInt();
JsonArray testGroups = test.getAsJsonArray("testGroups");
Summary
Cryptomator is a practical encryption tool for cloud storage, while Wycheproof is a comprehensive cryptographic test suite. Cryptomator offers a user-friendly solution but has a narrower focus, whereas Wycheproof provides extensive testing capabilities for various cryptographic implementations.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Project Wycheproof
Project Wycheproof is a community managed repository of test vectors that can be used by cryptography library developers to test against known attacks, specification inconsistencies, and other various implementation bugs.
Test vectors are maintained as JSON test vector data, with accompanying JSON schema files that document the structure of the test vector data.
Getting started
- Clone this repository. You may also want to integrate Wycheproof as a Git submodule or otherwise set up automation to keep track of changes over time.
- Write (or generate from the JSON schema files) code to load the vector data as appropriate for your implementation language/project.
- For each algorithm of interest, identify the inputs to your cryptography APIs, and the produced outputs, mapping back to what the test vectors provide.
- Iterate through applicable test vectors, ensuring that the results produced by your API when given the relevant input data matches the test vector expected results.
- For best results, integrate this process into your continuous integration (CI) process so tests are run for all new contributions/changes.
You may find it helpful to examine how other projects like pyca/cryptography have integrated Wycheproof's test vectors.
Coverage
Project Wycheproof has test vectors for the most popular crypto algorithms, including
- AEGIS
- AES-EAX
- AES-FF1
- AES-GCM
- AES-SIV
- ARIA
- ASCON
- Camellia
- ChaCha20-Poly1305
- XChaCha20-Poly1305
- DH
- DHIES
- DSA
- ECDH
- ECDSA
- EdDSA
- ECIES
- HKDF
- HMAC
- KMAC
- MORUS
- PBKDF2
- RSA
- SEED
- SipHash
- SM4
- VMAC
- X25519, X448
- ML-KEM (Kyber)
- ML-DSA (CRYSTALS-Dilithium)
The test vectors detect whether a library is vulnerable to many attacks, including
- Invalid curve attacks
- Biased nonces in digital signature schemes
- Of course, all Bleichenbacherâs attacks
- And many more -- we have over 80 test cases
We welcome contribution of new test vector data, and algorithms.
Contributing
If you want to contribute, please read CONTRIBUTING and send us pull requests. You can also report bugs or request new tests as GitHub issues.
Development Priorities
We're in the process of revitalizing development and maintenance of Project Wycheproof as a C2SP project with a renewed focus on the test vector data. Our immediate priorities are:
- Completing JSON schema descriptions of all test vectors.
- Improving documentation and tooling support for external contributors to provide new test data to existing vector files.
- Developing a community of downstream consumers who can help sheppard maintenance and review of new test vector data.
- Adding additional algorithm and test case coverage to the test vector data.
FAQ
Why is the project called "Wycheproof"?
Project Wycheproof is named after Mount Wycheproof, the smallest mountain in the world. The main motivation for the project at the time of its creation was to have a goal that is achievable. The smaller the mountain the more likely it is to be able to climb it.
What downstream projects use Wycheproof testvectors?
Wycheproof test vectors are used in some form by a number of important cryptography projects and libraries. In no particular order these include:
- OpenSSL
- BoringSSL
- aws-lc
- LibreSSL
- NSS
- pyca/cryptography
- Botan
- Go cryptography
- swift-crypto
- RustCrypto
- Graviola
- Tink
- PyCryptdome
- OpenTitan
- Zig
If your project uses test vectors from Wycheproof, feel free to open a PR to add it to the list above!
Has Wycheproof testing found notable bugs?
See doc/bugs.md for some notable historic bugs found using Wycheproof's test harnesses, or test vector data.
Where is the testvectors/ directory?
We recently combined the testvectors/ and testvectors_v1/ directories into
a single unified directory with one consistent approach to schemas.
Users requiring the original ("v0") test vector data can clone this repo from
the wycheproof-v0-vectors tag, but are encouraged to consider updating to
use testvectors_v1/ to benefit from future updates. If there are features/test
coverage from testvectors/ missing from testvectors_v1/, or there's another
issue blocking your update please open an issue describing your needs.
Do all vectors have schemas?
At the time of writing, the following testvectors_v1 files are missing schemas:
testvectors_v1/aes_ff1_base*_test.jsontestvectors_v1/aes_ff1_radix*_test.jsontestvectors_v1/ecdsa_secp256k1_sha256_bitcoin_test.jsontestvectors_v1/pbes2_hmacsha*_aes_*_test.jsontestvectors_v1/pbkdf2_hmacsha*_test.jsontestvectors_v1/rsa_pss_*_sha*_mgf*_params_test.jsontestvectors_v1/rsa_pss_misc_params_test.json
Contribution of schemas for the above vectors would be most welcome.
Is there additional documentation about test vectors?
Some legacy documentation for files, formats and types are available, but not necessarily in-sync with the current test vector state.
In general, prefer referencing the schema files since these are tested in CI to ensure vector file contents match their advertised schema.
Where is the test harness code?
Historically Wycheproof also included test harnesses (e.g. for Java and
Javascript cryptography implementations) that tested a variety of attacks
directly against implementations. Since transitioning to community support
these harnesses have been removed (but still exist in git
history for interested parties at cd27d64). Our current focus is on
implementation-agnostic test vectors.
Testing 3rd party cryptography libraries directly means flaws are only uncovered after they have been committed, and potentially released, by the projects under test. Instead, we encourage downstream projects to regularly test their code using Wycheproof test vectors as part of their development process. This approach helps catch flaws before they can become CVEs, means new features get tested immediately, and helps distribute the maintenance burden. This allows the Wycheproof maintainers to focus on test vectors instead of tracking downstream development of many projects while simultaneously maintaining an ever-increasing number of language & project-specific test harnesses.
Parties interested in test harnesses may find continued work by Daniel Bleichenbacher in Rooterberg of interest.
Who created Wycheproof?
Project Wycheproof was originally created and maintained by:
- Daniel Bleichenbacher
- Thai Duong
- Emilia Kasper
- Quan Nguyen
- Charles Lee
Top Related Projects
Project Wycheproof tests crypto libraries against known attacks.
Guidelines for low-level cryptography software
A curated list of cryptography resources and links.
Cryptomator for Windows, macOS, and Linux: Secure client-side encryption for your cloud storage, ensuring privacy and control over your data.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot