SHA3系列(KECCAK)哈希算法原理及实现(附源码)

大家好，欢迎来到IT知识分享网。

相关文章: (本文持续更新中)

• SHA3系列(KECCAK)哈希算法原理及实现(附源码)
• SHA512系列哈希算法原理及实现(附源码)
• SHA224和SHA256哈希算法原理及实现(附源码)
• 国密SM3哈希算法原理及实现(附源码)
• SHA1哈希算法原理及实现(附源码)
• MD5哈希算法原理及实现(附源码)
• MD4哈希算法原理及实现(附源码)
• MD2哈希算法原理及实现(附源码)
• MD2中用于随机置换的S盒是如何生成的？

1. SHA3来源

在2004-2005年，对哈希函数MD5和SHA-0的碰撞研究出现了突破。其中包括当时还在山东大学的王小云教授所在的团队(王小云、冯登国、来学嘉、于红波)，使用差分分析的办法，对多个哈希函数(MD4、MD5、HAVAL-128和RIPEMD)进行有效的破译。根据维基百科的描述，使用这个方法，在IBM p690服务器上，一小时就可以根据现有消息的MD5哈希值找到具有同样哈希的另一条消息。

虽然SHA-1暂时还未被攻破，即还没有出现在实际运算能力之内短时间寻找碰撞的方法。但是SHA-1在设计上与MD5和SHA-0具有相似的结构和基本数学运算，而后两者已经被攻破。因此SHA-1 被认为是不安全的，应逐步被SHA-2取代。

根据维基百科SHA-1词条的描述，到2020年，针对SHA-1的chosen-prefix attack已经变得可行，所以SHA-1也变得不安全了。

对于SHA-2，特别是512位版本的安全强度对目前的计算能力来说还是足够的。但是SHA-2同样和它的上一版一样在设计上有类似的结构和基本数学运算。可以预见，在若干年后SHA-2的缺陷也会被发现而被取代，因此NIST决定开展新的HASH标准的制定工作。

截止到2008年10月31日，NIST一共收到64个算法：

• 第一轮筛选(12/10/2008-7/24/2009), NIST在2008年12月9日公布了51个候选算法，2009年7月24公布了第一轮胜出的14个算法。
• 第二轮筛选(9/28/2009-12/9/2010), NIST在2009年12月9日公布了第二轮胜出的5个算法。
• 第三轮筛选(1/31/2011-10/2/2012), NIST在2012年10月2日宣布Keccak在第三轮筛选中胜出，将作为SHA-3标准。
• 最终，NIST在2015年8月5日颁布了SHA-3标准FIPS-202。

在最初提交的64个算法中，也包含MD2/MD4/MD5家族设计者Rivest提交的MD6算法，不过该算法在第一轮中被淘汰，没能进入第二轮。在参与第三轮最终竞争的5个算法中，有一个名叫JH的算法，作者是Hongjun Wu, 来自新加坡理工大学，从名字看是位华裔。胜出的Keccak算法由一个来自比利时和意大利的密码团队设计，设计者之一的Joan Daemen也是2001年公布的AES算法的两个设计者之一。

2. SHA3和SHA2的关系

目前SHA-2发展得很好，而且NIST认为它对于一般应用是安全的，所以SHA-3对于SHA-2是一种补充或者备用方案而不是替代。

因此，SHA-3标准FIPS-202是对此前标准FIPS-180-4的补充，而不是我们理解的代替关系。(我自己以前就将SHA3理解为替代SHA2的下一代哈希标准)。

3. 关于本文

最近陆续造了一批哈希算法的轮子，包括MD家族(包括MD2/MD4/MD5), SHA1, SHA2家族(SHA224, SHA256, SHA384, SHA512)，SHA3家族(SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256)以及国密SM3算法。

原来打算将每一个算法都详细分析并实现，现在看来，这个工作短时间可能无法完成，所以先将源码发上来。

最近陆续造了一批哈希算法的轮子，包括MD家族(包括MD2/MD4/MD5), SHA1, SHA2家族(SHA224, SHA256, SHA384, SHA512)，SHA3家族(SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256)以及国密SM3算法。

原来打算将每一个算法都详细分析并实现，现在看来，这个工作短时间可能无法完成，所以先将源码发上来。

这部分实现的源码完全参考官方文档FIPS-202的算法描述，连变量名也尽可能和官方文档中的变量保持一致，方便学习。

代码除了实现常用的SHA3算法SHA3-224,SHA3-256,SHA3-384,SHA3-512外，还实现了扩展的两个SHA3 XOF算法SHAKE128和SHAKE256，支持任意字节长度的输出，并通过了检查验证，功能无误。

本篇主要是描述SHA3系列哈希算法的原理及实现，SHA3系列的哈希函数都是基于KECCAK哈希算法扩展而来，这些列主要包括:

• 4个定长输出的SHA3哈希函数: SHA3-224, SHA3-256, SHA3-384, SHA3-512，用于替代已经存在的SHA2系列函数
• 2个扩展SHA3函数(XOF-Extendable Output Function)，其输出的哈希值可以任意指定长度

对于SHA3 XOF函数SHAKE128和SHAKE256，最新的openssl v1.1.1已经支持，但只能输出指定长度值:

• SHAKE128, openssl工具输出长度为128 bits (16字节)
• SHAKE256, openssl工具输出长度为256 bits (32字节)

由于openssl没有专门针对SHA3函数提供接口，本文的SHA3系列函数的API封装调用接口参考了openssl SHA2系列函数的接口，会使用这里的接口，就会使用openssl的库函数接口。

除了实现的源码外，还另外附带了一个测试例子，这个测试例子不仅仅是用于测试哈希算法的实现是否正确，还可以提供了多个选项用于对任意文件和字符串进行哈希，因此作为一个工具使用，类似系统内置的md5sum/sha1sum。

SHA3的实现源码

1. 头文件sha3.c

/* * @ file: sha512.h * @ description: header file for sha3.c * @ author: Gu Yongqiang * @ blog: https://blog.csdn.net/guyongqiangx */ #ifndef __ROCKY_SHA3__H #define __ROCKY_SHA3__H #define ERR_OK 0 #define ERR_ERR -1 /* generic error */ #define ERR_INV_PARAM -2 /* invalid parameter */ #define ERR_TOO_LONG -3 /* too long */ #define ERR_STATE_ERR -4 /* state error */ typedef unsigned char uint8_t; typedef unsigned short uint16_t; typedef unsigned int uint32_t; typedef unsigned long long uint64_t; typedef struct { uint64_t high; /* high 64 bits */ uint64_t low; /* low 64 bits */ } uint128_t; /* * Standard: * SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf * * Understanding-Cryptography-Keccak.pdf * SHA-3 and The Hash Function Keccak * https://www.crypto-textbook.com/download/Understanding-Cryptography-Keccak.pdf */ typedef enum sha3_algorithm { SHA3_224, SHA3_256, SHA3_384, SHA3_512, SHAKE128, SHAKE256 }SHA3_ALG; typedef struct sha3_context { /* intermedia hash value for each block */ uint64_t lane[5][5]; /* 5 x 5 x 64 = 1600 bits */ /* last block */ struct { uint32_t used; /* used bytes */ uint8_t buf[200]; /* block data buffer, 200 x 8 = 1600 bits */ }last; SHA3_ALG alg; /* * |-------------------------------------------------------------| * | l | 0 | 1 | 2 | 3 | 4 | 5 | 6 | * |-------------------------------------------------------------| * | w = 2^l | 1 | 2 | 4 | 8 | 16 | 32 | 64 | * |-------------------------------------------------------------| * | b = 25*2^l | 25 | 50 | 100 | 200 | 400 | 800 | 1600 | * |-------------------------------------------------------------| * | SHA3: l = 6, w = 64, b = 1600 * | * |-------------------------------------------------------------| */ // uint32_t l; /* binary logarithm of lane size */ // uint32_t w; /* lane size in bits */ uint32_t b; /* width of the state, b = r + c */ uint32_t r; /* bit rate, rate of a sponge function, length of one message block */ uint32_t c; /* capacity, r + c = b */ uint32_t nr; /* round number, nr = 12 + 2l */ uint32_t md_size; /* message digest size in bytes */ uint32_t absorbing; /* 1: absorbe; 0: squeeze */ }SHA3_CTX; int SHA3_Init(SHA3_CTX *c, SHA3_ALG alg); int SHA3_Update(SHA3_CTX *c, const void *data, size_t len); int SHA3_Final(unsigned char *md, SHA3_CTX *c); unsigned char *SHA3(SHA3_ALG alg, const unsigned char *data, size_t n, unsigned char *md); /* Extendable-Output Functions: SHAKE128, SHAKE256 */ int SHA3_XOF_Init(SHA3_CTX *c, SHA3_ALG alg, uint32_t md_size); int SHA3_XOF_Update(SHA3_CTX *c, const void *data, size_t len); int SHA3_XOF_Final(unsigned char *md, SHA3_CTX *c); unsigned char *SHA3_XOF(SHA3_ALG alg, const unsigned char *data, size_t n, unsigned char *md, uint32_t md_size); #endif 

2. 代码文件sha3.c

/* * @ file: sha3.c * @ description: implementation for the SHA3 Secure Hash Algorithm * @ author: Gu Yongqiang * @ blog: https://blog.csdn.net/guyongqiangx */ #include <stdio.h> #include <string.h> #include "utils.h" #include "sha3.h" //#define DEBUG #ifdef DEBUG #define DBG(...) printf(__VA_ARGS__) #define DUMP_BLOCK_DATA 1 #define DUMP_BLOCK_HASH 1 #define DUMP_ROUND_DATA 1 #define DUMP_SCHED_DATA 1 #else #define DBG(...) #define DUMP_BLOCK_DATA 0 #define DUMP_BLOCK_HASH 0 #define DUMP_ROUND_DATA 0 #define DUMP_SCHED_DATA 0 #endif /* * FIPS-202, sec B.2: * |---------------|------------------------| * | Padding Bytes | Padding Message | * |---------------|------------------------| * | q=1 | M||0x86 | * |---------------|------------------------| * | q=2 | M||0x0680 | * |---------------|------------------------| * | q>2 | M||0x06||0x00...||0x80 | * |---------------|------------------------| * * refer: * https://cryptologie.net/article/387/byte-ordering-and-bit-numbering-in-keccak-and-sha-3/ */ /* * SHA3 Delimiter + Padding * 01 + 10*1 */ /* q=1: 01 10 0001 <--reverse-- 1000 01 10, 1 byte, 0x86 */ #define SHA3_PADDING_STD1 0x86 /* q>=2: 01 10 0000....0000 0001 <--reverse-- 0000 01 10....1000 0000, 2 bytes, 0x06...0x80 */ #define SHA3_PADDING_STD2_BEGIN 0x06 #define SHA3_PADDING_STD2_END 0x80 /* * SHA3 XOF Delimiter + Padding * 1111 + 10*1 */ /* q=1: 1111 1001 <--reverse-- 1001 1111, 1 byte, 0x9F */ #define SHA3_PADDING_XOF1 0x9F /* q>=2: 1111 1000....0000 0001 <--reverse 0001 1111....1000 0000, 2 bytes, 0x1F...0x80 */ #define SHA3_PADDING_XOF2_BEGIN 0x1F #define SHA3_PADDING_XOF2_END 0x80 /* ROTate Left (circular left shift) */ static uint64_t ROTL(uint64_t x, uint8_t shift) { return (x << shift) | (x >> (64 - shift)); } static uint32_t theta(uint64_t A[5][5]) { uint32_t x, y; uint64_t Ap[5][5]; uint64_t C[5], D[5]; memset(C, 0, sizeof(C)); memset(D, 0, sizeof(D)); memset(Ap, 0, sizeof(Ap)); for (x=0; x<5; x++) { C[x] = A[0][x] ^ A[1][x] ^ A[2][x] ^ A[3][x] ^ A[4][x]; } for (x=0; x<5; x++) { D[x] = C[(x+4)%5] ^ ROTL(C[(x+1)%5], 1); } for (y=0; y<5; y++) { for (x=0; x<5; x++) { Ap[y][x] = A[y][x] ^ D[x]; } } memcpy(A, Ap, sizeof(Ap)); return 0; } /* rotation constants, aka rotation offsets */ static uint32_t Rp[5][5] = { { 0, 1, 190, 28, 91}, { 36, 300, 6, 55, 276}, { 3, 10, 171, 153, 231}, { 105, 45, 15, 21, 136}, { 210, 66, 253, 120, 78} }; static uint32_t rho(uint64_t A[5][5]) { uint64_t Ap[5][5]; uint32_t x, y, m; uint32_t t; memset(Ap, 0, sizeof(Ap)); /* let A'[0,0,z]=A[0,0,z] */ memcpy(Ap[0], A[0], sizeof(Ap[0])); /* let (x,y) = (1,0) */ x = 1; y = 0; #if 0 /* calculate directly */ for (t=0; t<24; t++) { Ap[y][x] = ROTL(A[y][x], ((t+1)*(t+2)/2)%64); m = x; x = y; y = (2*m + 3*y) % 5; } #else /* look up table */ for (t=0; t<24; t++) { Ap[y][x] = ROTL(A[y][x], Rp[y][x]%64); /* let (x,y) = (y,(2x+3y)%5) */ m = x; x = y; y = (2*m+3*y) % 5; } #endif memcpy(A, Ap, sizeof(Ap)); return 0; } static uint32_t pi(uint64_t A[5][5]) { uint64_t Ap[5][5]; uint32_t x, y; memset(Ap, 0, sizeof(Ap)); for (y=0; y<5; y++) { for (x=0; x<5; x++) { Ap[y][x] = A[x][(x+3*y)%5]; } } memcpy(A, Ap, sizeof(Ap)); return 0; } static uint32_t chi(uint64_t A[5][5]) { uint64_t Ap[5][5]; uint32_t x, y; memset(Ap, 0, sizeof(Ap)); for (y=0; y<5; y++) { for (x=0; x<5; x++) { Ap[y][x] = A[y][x] ^ ((~A[y][(x+1)%5]) & A[y][(x+2)%5]); } } memcpy(A, Ap, sizeof(Ap)); return 0; } static uint64_t RC[24] = { 0x0000000000000001, 0x0000000000008082, 0x0808a, 0x08000, 0x000000000000808b, 0x00000000, 0x08081, 0x08009, 0x000000000000008a, 0x0000000000000088, 0x00000000, 0x00000000a, 0x00000000b, 0x0008b, 0x08089, 0x08003, 0x08002, 0x00080, 0x000000000000800a, 0x0000a, 0x08081, 0x08080, 0x00000000, 0x08008, }; static uint32_t iota(uint64_t A[5][5], uint32_t i) { A[0][0] = A[0][0] ^ RC[i]; return 0; } int SHA3_Init(SHA3_CTX *c, SHA3_ALG alg) { if (NULL == c) { return ERR_INV_PARAM; } if ((alg == SHAKE128) || (alg == SHAKE256)) { return ERR_INV_PARAM; } memset(c, 0, sizeof(SHA3_CTX)); /* bits */ // c->l = 6; // c->w = 64; /* c->w = 2 ^ l */ /* bytes */ c->b = 200; /* 1600 bits, c->b = 25 * 2 ^ c->l; */ c->alg = alg; switch (alg) { case SHA3_224: /* SHA3-224(M) = KECCAK[448](M||01,224), FIPS-202, sec 6.1 */ c->r = 144; /* 1152 bits */ c->c = 56; /* 448 bits */ c->md_size = 28; /* 224 bits */ break; case SHA3_256: /* SHA3-256(M) = KECCAK[512](M||01,256), FIPS-202, sec 6.1 */ c->r = 136; /* 1088 bits */ c->c = 64; /* 512 bits */ c->md_size = 32; /* 256 bits */ break; case SHA3_384: /* SHA3-384(M) = KECCAK[768](M||01,384), FIPS-202, sec 6.1 */ c->r = 104; /* 832 bits */ c->c = 96; /* 768 bits */ c->md_size = 48; /* 384 bits */ break; default: /* default Keccak setting: SHA3_512 */ case SHA3_512: /* SHA3-512(M) = KECCAK[1024](M||01,512), FIPS-202, sec 6.1 */ c->r = 72; /* 576 bits */ c->c = 128; /* 1024 bits */ c->md_size = 64; /* 512 bits */ break; } c->nr = 24; /* nr = 24 = 12 + 2 * l */ c->absorbing = 1; /* absorbing phase */ return ERR_OK; } #if (DUMP_SCHED_DATA == 1) #define sched_show_buffer(info,ptr,len) \ DBG(info); \ print_buffer((ptr),(len)," "); #else #define sched_show_buffer(info,ptr,len) #endif #if (DUMP_ROUND_DATA == 1) #define round_show_buffer(info) \ DBG(info); \ print_buffer(&ctx->lane[0][0], ctx->b, " "); static void dump_lane_buffer(uint64_t lane[5][5]) { uint32_t x, y; for (y=0; y<5; y++) /* row */ { for (x=0; x<5; x++) /* col */ { DBG("[%d, %d]: %016llx ", x, y, lane[y][x]); } DBG("\n"); } return; } #else #define round_show_buffer(info) static void dump_lane_buffer(uint64_t lane[5][5]) {} #endif static int SHA3_PrepareScheduleWord(SHA3_CTX *ctx, const uint64_t *block) { uint32_t i; uint64_t *data; uint64_t temp[25]; if ((NULL == ctx) || (NULL == block)) { return ERR_INV_PARAM; } for (i=0; i<ctx->b/8; i++) { if (i<ctx->r/8) { temp[i] = le64toh(block[i]); } else { temp[i] = 0x0; } } sched_show_buffer("Data to absorbed:\n", temp, ctx->b); sched_show_buffer(" SchedWord: [before]\n", &ctx->lane[0][0], ctx->b); /* initial S */ data = &ctx->lane[0][0]; for (i=0; i<ctx->b/8; i++) { data[i] ^= temp[i]; } sched_show_buffer(" SchedWord: [after]\n", &ctx->lane[0][0], ctx->b); return ERR_OK; } /* r bytes for each block */ static int SHA3_ProcessBlock(SHA3_CTX *ctx, const void *block) { uint32_t t; if ((NULL == ctx) || (ctx->absorbing && (NULL == block))) { return ERR_INV_PARAM; } #if (DUMP_BLOCK_DATA == 1) DBG("---------------------------------------------------------\n"); DBG(" BLOCK DATA:\n"); print_buffer(block, ctx->r, " "); #endif if (ctx->absorbing) { SHA3_PrepareScheduleWord(ctx, block); } for (t=0; t<ctx->nr; t++) { #if (DUMP_ROUND_DATA == 1) DBG(" Round #%02d:\n", t); #endif theta(ctx->lane); round_show_buffer("After Theta:\n"); rho(ctx->lane); round_show_buffer(" After Rho:\n"); pi(ctx->lane); round_show_buffer(" After Pi:\n"); chi(ctx->lane); round_show_buffer(" After Chi:\n"); iota(ctx->lane, t); round_show_buffer(" After Iota:\n"); } round_show_buffer("After Permutation:\n"); #if (DUMP_BLOCK_HASH == 1) DBG(" BLOCK HASH:\n"); print_buffer(&ctx->lane[0][0], ctx->b, " "); #endif return ERR_OK; } int SHA3_Update(SHA3_CTX *c, const void *data, size_t len) { uint64_t copy_len = 0; if ((NULL == c) || (NULL == data)) { return ERR_INV_PARAM; } /* has used data */ if (c->last.used != 0) { /* less than 1 block in total, combine data */ if (c->last.used + len < c->r) { memcpy(&c->last.buf[c->last.used], data, len); c->last.used += len; return ERR_OK; } else /* more than 1 block */ { /* process the block in context buffer */ copy_len = c->r - c->last.used; memcpy(&c->last.buf[c->last.used], data, copy_len); SHA3_ProcessBlock(c, &c->last.buf); data = (uint8_t *)data + copy_len; len -= copy_len; /* reset context buffer */ memset(&c->last.buf[0], 0, c->r); c->last.used = 0; } } /* less than 1 block, copy to context buffer */ if (len < c->r) { memcpy(&c->last.buf[c->last.used], data, len); c->last.used += len; return ERR_OK; } else { /* process data blocks */ while (len >= c->r) { SHA3_ProcessBlock(c, data); data = (uint8_t *)data + c->r; len -= c->r; } /* copy rest data to context buffer */ memcpy(&c->last.buf[0], data, len); c->last.used = len; } return ERR_OK; } int SHA3_Final(unsigned char *md, SHA3_CTX *c) { uint32_t md_size = 0; /* message digest size used */ if ((NULL == c) || (NULL == md)) { return ERR_INV_PARAM; } /* more than 2 bytes left */ if (c->last.used <= (c->r - 2)) { /* one more block */ if ((c->alg == SHAKE128) || (c->alg == SHAKE256)) /* XOFs */ { c->last.buf[c->last.used] = SHA3_PADDING_XOF2_BEGIN; } else { c->last.buf[c->last.used] = SHA3_PADDING_STD2_BEGIN; } c->last.used++; memset(&c->last.buf[c->last.used], 0, (c->r - 1) - c->last.used); c->last.used = c->r - 1; if ((c->alg == SHAKE128) || (c->alg == SHAKE256)) /* XOFs */ { c->last.buf[c->last.used] = SHA3_PADDING_XOF2_END; } else { c->last.buf[c->last.used] = SHA3_PADDING_STD2_END; } c->last.used++; } else /* if (c->last.used == (c->r - 1)) */ /* only 1 bytes left */ { if ((c->alg == SHAKE128) || (c->alg == SHAKE256)) /* XOFs */ { c->last.buf[c->last.used] = SHA3_PADDING_XOF1; } else { c->last.buf[c->last.used] = SHA3_PADDING_STD1; } c->last.used++; } SHA3_ProcessBlock(c, &c->last.buf); c->last.used = 0; /* Absorbing Phase End */ c->absorbing = 0; dump_lane_buffer(c->lane); if (c->md_size <= c->r) { memcpy(md, &c->lane[0][0], c->md_size); } else { memcpy(md, &c->lane[0][0], c->r); md_size = c->r; /* squeeze */ while (md_size < c->md_size) { SHA3_ProcessBlock(c, NULL); if (c->md_size - md_size > c->r) { memcpy(&md[md_size], &c->lane[0][0], c->r); md_size += c->r; } else { memcpy(&md[md_size], &c->lane[0][0], c->md_size - md_size); md_size = c->md_size; } } } return ERR_OK; } unsigned char *SHA3(SHA3_ALG alg, const unsigned char *data, size_t n, unsigned char *md) { SHA3_CTX c; if ((NULL == data) || (NULL == md)) { return NULL; } SHA3_Init(&c, alg); SHA3_Update(&c, data, n); SHA3_Final(md, &c); return md; } int SHA3_XOF_Init(SHA3_CTX *c, SHA3_ALG alg, uint32_t md_size) { if (NULL == c) { return ERR_INV_PARAM; } /* only for SHAKE128/SHAKE256 */ if ((alg != SHAKE128) && (alg != SHAKE256)) { return ERR_INV_PARAM; } /* using SHA3-512 as default */ SHA3_Init(c, SHA3_512); c->alg = alg; /* update for SHAKE128/SHAKE256 */ switch(alg) { case SHAKE128: /* SHAKE128(M,d) = KECCAK[256](M||1111,d), FIPS-202, sec 6.2 */ c->r = 168; /* 1344 bits */ c->c = 32; /* 256 bits */ c->md_size = md_size; break; default: case SHAKE256: /* SHAKE256(M,d) = KECCAK[512](M||1111,d), FIPS-202, sec 6.2 */ c->r = 136; /* 1088 bits */ c->c = 64; /* 512 bits */ c->md_size = md_size; break; } return ERR_OK; } int SHA3_XOF_Update(SHA3_CTX *c, const void *data, size_t len) { return SHA3_Update(c, data, len); } int SHA3_XOF_Final(unsigned char *md, SHA3_CTX *c) { return SHA3_Final(md, c); } unsigned char *SHA3_XOF(SHA3_ALG alg, const unsigned char *data, size_t n, unsigned char *md, uint32_t md_size) { SHA3_CTX c; if ((NULL == data) || (NULL == md)) { return NULL; } /* only for SHAKE128/SHAKE256 */ if ((alg != SHAKE128) && (alg != SHAKE256)) { return NULL; } SHA3_XOF_Init(&c, alg, md_size); SHA3_XOF_Update(&c, data, n); SHA3_XOF_Final(md, &c); return md; } 

从上面的实现来看，SHA3 XOF函数和普通SHA3函数的主要区别在于：

• 普通SHA3函数的输出是固定的，哈希算法指定以后出处的哈希值长度就固定了
• 扩展的SHA3 XOF函数的输出可以是任意长度，所以在初始化(SHA3_XOF_Init)和直接操作SHA3_XOF时需要额外的参数指定输出的哈希长度

另外，普通的SHA3函数处理时，吸水阶段(Absorbing)结束后直接返回哈希值，扩展的SHA3 XOF函数如果输出长度大于当前的处理长度时，还需要进入挤压(Squeezing)阶段。

SHA3源码的编译和测试

我直接在Makefile中内置了一个test伪目标，编译时除了编译生成名为sha3的哈希工具外，还会直接调用内置的哈希测试。

编译和运行如下：

sha3$ make gcc -Wall -g -O2 -c utils.c -o utils.o gcc -Wall -g -O2 -c sha3.c -o sha3.o gcc -Wall -g -O2 -c sha3test.c -o sha3test.o gcc -Wall -g -O2 utils.o sha3.o sha3test.o -o sha3 Run Test... ./sha3 -a sha3-224 -x Internal hash tests for sha3-224(SHA3-224): sha3-224("") Expect: 6b4e0dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7 Result: 6b4e0dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7 sha3-224("a") Expect: 9e86ff69557ca95f405f0b38e3a819b309ee942f482b6a8b Result: 9e86ff69557ca95f405f0b38e3a819b309ee942f482b6a8b sha3-224("abc") Expect: ec3f8cf24ad09234ee7d3c766fc9a3a5168d0c94ad73b46fdf Result: ec3f8cf24ad09234ee7d3c766fc9a3a5168d0c94ad73b46fdf sha3-224("message digest") Expect: 18768bb4c48eb7fc88e5ddb17efcf2964abd7798a39d86a4b4a1e4c8 Result: 18768bb4c48eb7fc88e5ddb17efcf2964abd7798a39d86a4b4a1e4c8 sha3-224("abcdefghijklmnopqrstuvwxyz") Expect: 5cdeca81e123f87cad96b9cba999f16f6dd4e0f4681b8239 Result: 5cdeca81e123f87cad96b9cba999f16f6dd4e0f4681b8239 sha3-224("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: a67c289b8250a6f437ada8c163d45261be Result: a67c289b8250a6f437ada8c163d45261be sha3-224("890") Expect: 0ef91b3e2a76dd72a15dc6940a67c8164a044cd25cc8 Result: 0ef91b3e2a76dd72a15dc6940a67c8164a044cd25cc8 ./sha3 -a sha3-256 -x Internal hash tests for sha3-256(SHA3-256): sha3-256("") Expect: a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a Result: a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a sha3-256("a") Expect: 80084bf2fba0feb2cab2d8215eab14bc6bdd8bfb2cecd8b Result: 80084bf2fba0feb2cab2d8215eab14bc6bdd8bfb2cecd8b sha3-256("abc") Expect: 3a985da74fe225b2045c172d6bd390bd855f086e3e9d525b46bfe Result: 3a985da74fe225b2045c172d6bd390bd855f086e3e9d525b46bfe sha3-256("message digest") Expect: edcdbec18c3a11465eca34bce6143d30c8665cefcfd32bffd Result: edcdbec18c3a11465eca34bce6143d30c8665cefcfd32bffd sha3-256("abcdefghijklmnopqrstuvwxyz") Expect: 7cab2dc765e21b241dbc1c255ce620b29f527c6d5e7f5f843e56288f0d Result: 7cab2dc765e21b241dbc1c255ce620b29f527c6d5e7f5f843e56288f0d sha3-256("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: a79d6a9da47f04a3b9a9323ec9991f2105d4c78a7bc7beeba7a11dfb9f Result: a79d6a9da47f04a3b9a9323ec9991f2105d4c78a7bc7beeba7a11dfb9f sha3-256("890") Expect: 293e5ce4ce54ee71990ab06e511b7ccd62722b1beb414f5ff65c8274e0f5be1d Result: 293e5ce4ce54ee71990ab06e511b7ccd62722b1beb414f5ff65c8274e0f5be1d ./sha3 -a sha3-384 -x Internal hash tests for sha3-384(SHA3-384): sha3-384("") Expect: 0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2acadb47fb6bd1e058d5f004 Result: 0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2acadb47fb6bd1e058d5f004 sha3-384("a") Expect: 1815f774fb48569efec794d249eeb59aae46d22bf77dafe25c5edc28d7ea44f93ee1234aa88f61c91912a4ccd9 Result: 1815f774fb48569efec794d249eeb59aae46d22bf77dafe25c5edc28d7ea44f93ee1234aa88f61c91912a4ccd9 sha3-384("abc") Expect: ec0fcf58e2c6ad8df9b473cb0fc08c2596da7cf0e49be4b298d88cea927ac7f539f1edfd25 Result: ec0fcf58e2c6ad8df9b473cb0fc08c2596da7cf0e49be4b298d88cea927ac7f539f1edfd25 sha3-384("message digest") Expect: df44af73e2c8ea979de3d61dc02bf69def7fbffdfffef19ad57e17d4b93ba1e484fc1980d5 Result: df44af73e2c8ea979de3d61dc02bf69def7fbffdfffef19ad57e17d4b93ba1e484fc1980d5 sha3-384("abcdefghijklmnopqrstuvwxyz") Expect: fed399d2217aaf4c717ad0c5102c15589e1c990cc2b9aa7fd6ab65dba5cadb53fc9280d278f Result: fed399d2217aaf4c717ad0c5102c15589e1c990cc2b9aa7fd6ab65dba5cadb53fc9280d278f sha3-384("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: d5bf5080d0830e0de7b6b2cfa008f4c4f386ac742d20cb45aa51bd4f542fc733e2719e Result: d5bf5080d0830e0de7b6b2cfa008f4c4f386ac742d20cb45aa51bd4f542fc733e2719e sha3-384("890") Expect: 3c213a17facb3bf17f109f3e24c16f9f14f085b52a2f2b81adc0db83df1a58db2ce013191b8ba72d8fae7e2a5e Result: 3c213a17facb3bf17f109f3e24c16f9f14f085b52a2f2b81adc0db83df1a58db2ce013191b8ba72d8fae7e2a5e ./sha3 -a sha3-512 -x Internal hash tests for sha3-512(SHA3-512): sha3-512("") Expect: a69f73cca23a9ac5c8b567dc185a756e97cfe25859e0d1dcc1475c80a615b2123af1f5f94c11e3e9402c3ac558fd95b6d3e1dcd26 Result: a69f73cca23a9ac5c8b567dc185a756e97cfe25859e0d1dcc1475c80a615b2123af1f5f94c11e3e9402c3ac558fd95b6d3e1dcd26 sha3-512("a") Expect: 697f2dcb8309d6b8b97dac4de344b549d4dee61edfb4962d8698b7fa803f4f93ffe28b5b957ac3d1dcef997bd336d09ab02a Result: 697f2dcb8309d6b8b97dac4de344b549d4dee61edfb4962d8698b7fa803f4f93ffe28b5b957ac3d1dcef997bd336d09ab02a sha3-512("abc") Expect: bb1a57168a5693cd924b6b096e08ff70d884f5d0240d2712e10e116e9192af3c91a7ec57647eb4cf408d5a56592f8274eec53f0 Result: bb1a57168a5693cd924b6b096e08ff70d884f5d0240d2712e10e116e9192af3c91a7ec57647eb4cf408d5a56592f8274eec53f0 sha3-512("message digest") Expect: 3444efa15511f57726c7d7cfe80302ab29d59a71415ca9dd141ac892d310bc4d78128c98fda839d18d7f0556f2fe7acb3c0cda4bff3a25f5f59 Result: 3444efa15511f57726c7d7cfe80302ab29d59a71415ca9dd141ac892d310bc4d78128c98fda839d18d7f0556f2fe7acb3c0cda4bff3a25f5f59 sha3-512("abcdefghijklmnopqrstuvwxyz") Expect: af328d17fa28753a3c9f5cb72e376b90440b96f0289e5703ba975ab384eda565fc92aadeddacdc0a5ffa358bd0571aaad80aca68 Result: af328d17fa28753a3c9f5cb72e376b90440b96f0289e5703ba975ab384eda565fc92aadeddacdc0a5ffa358bd0571aaad80aca68 sha3-512("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: d1db17b4745b255e5eb159f66593cc9cfc7aaba80165aab536b46174ce19e3f707f0e5c6487f5f03084bc0ecef20113e42ad28163 Result: d1db17b4745b255e5eb159f66593cc9cfc7aaba80165aab536b46174ce19e3f707f0e5c6487f5f03084bc0ecef20113e42ad28163 sha3-512("890") Expect: 9524b9a5536bb4f6196b7e9475b4da69e01f0cf224cd7335ddb286fd99b9b32ffe33b59ad424cc1744f6eb59137f5fbe8a8af0ae930 Result: 9524b9a5536bb4f6196b7e9475b4da69e01f0cf224cd7335ddb286fd99b9b32ffe33b59ad424cc1744f6eb59137f5fbe8a8af0ae930 ./sha3 -a shake128 -d 128 -x Internal hash tests for shake128(SHAKE128): shake128("") Expect: 7f9c2ba4e88f827d5853e Result: 7f9c2ba4e88f827d5853e shake128("a") Expect: 85c8de88d28866bf0bb Result: 85c8de88d28866bf0bb shake128("abc") Expect: dd818bf5cf8a3ddb793fbcba7 Result: dd818bf5cf8a3ddb793fbcba7 shake128("message digest") Expect: cbefb55b4cdf491 Result: cbefb55b4cdf491 shake128("abcdefghijklmnopqrstuvwxyz") Expect: 961c919c0ee81514bf37 Result: 961c919c0ee81514bf37 shake128("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: 54dd201edb3c7dfbb6 Result: 54dd201edb3c7dfbb6 shake128("890") Expect: 7bf451c92fdc77b9771e6c Result: 7bf451c92fdc77b9771e6c ./sha3 -a shake128 -x Internal hash tests for shake128(SHAKE128): shake128("") Expect: 7f9c2ba4e88f827d5853e Result: 7f9c2ba4e88f827d5853e shake128("a") Expect: 85c8de88d28866bf0bb Result: 85c8de88d28866bf0bb shake128("abc") Expect: dd818bf5cf8a3ddb793fbcba7 Result: dd818bf5cf8a3ddb793fbcba7 shake128("message digest") Expect: cbefb55b4cdf491 Result: cbefb55b4cdf491 shake128("abcdefghijklmnopqrstuvwxyz") Expect: 961c919c0ee81514bf37 Result: 961c919c0ee81514bf37 shake128("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: 54dd201edb3c7dfbb6 Result: 54dd201edb3c7dfbb6 shake128("890") Expect: 7bf451c92fdc77b9771e6c Result: 7bf451c92fdc77b9771e6c ./sha3 -a shake256 -d 256 -x Internal hash tests for shake256(SHAKE256): shake256("") Expect: 46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762f Result: 46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762f shake256("a") Expect: 867e2cb04f5a04dcbda5e8fe9ceaafcaca736cba4 Result: 867e2cb04f5a04dcbda5e8fe9ceaafcaca736cba4 shake256("abc") Expect: 0a8771ccc4114d8db44530f8f1e1ee4f94ea37e78b5739 Result: 0a8771ccc4114d8db44530f8f1e1ee4f94ea37e78b5739 shake256("message digest") Expect: 718e0ade4dc73487e15826a07ecb8ed5e2bda526cc1acddb99d00 Result: 718e0ade4dc73487e15826a07ecb8ed5e2bda526cc1acddb99d00 shake256("abcdefghijklmnopqrstuvwxyz") Expect: b7b78b04a3dd30a265c8886c33fdade5d3d10541fd4e9fc61 Result: b7b78b04a3dd30a265c8886c33fdade5d3d10541fd4e9fc61 shake256("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: 31f19a097c723e91fa59b0998dd8523c2a9e7e13b4025d6b48fcbca108 Result: 31f19a097c723e91fa59b0998dd8523c2a9e7e13b4025d6b48fcbca108 shake256("890") Expect: 24c508adefdf5e3f2596e8b5a888fe10eb7b5b22e1f35d858e6eff3025c4cc18 Result: 24c508adefdf5e3f2596e8b5a888fe10eb7b5b22e1f35d858e6eff3025c4cc18 ./sha3 -a shake256 -x Internal hash tests for shake256(SHAKE256): shake256("") Expect: 46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762f Result: 46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762f shake256("a") Expect: 867e2cb04f5a04dcbda5e8fe9ceaafcaca736cba4 Result: 867e2cb04f5a04dcbda5e8fe9ceaafcaca736cba4 shake256("abc") Expect: 0a8771ccc4114d8db44530f8f1e1ee4f94ea37e78b5739 Result: 0a8771ccc4114d8db44530f8f1e1ee4f94ea37e78b5739 shake256("message digest") Expect: 718e0ade4dc73487e15826a07ecb8ed5e2bda526cc1acddb99d00 Result: 718e0ade4dc73487e15826a07ecb8ed5e2bda526cc1acddb99d00 shake256("abcdefghijklmnopqrstuvwxyz") Expect: b7b78b04a3dd30a265c8886c33fdade5d3d10541fd4e9fc61 Result: b7b78b04a3dd30a265c8886c33fdade5d3d10541fd4e9fc61 shake256("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0") Expect: 31f19a097c723e91fa59b0998dd8523c2a9e7e13b4025d6b48fcbca108 Result: 31f19a097c723e91fa59b0998dd8523c2a9e7e13b4025d6b48fcbca108 shake256("890") Expect: 24c508adefdf5e3f2596e8b5a888fe10eb7b5b22e1f35d858e6eff3025c4cc18 Result: 24c508adefdf5e3f2596e8b5a888fe10eb7b5b22e1f35d858e6eff3025c4cc18 ./sha3 -a sha3-224 -f sha3 sha3-224(sha3) = 5cd66184d2174deb961f2b4be3e20dcdb1ced59a12e ./sha3 -a sha3-256 -f sha3 sha3-256(sha3) = 40e73c1c29ea8e6093dd31fa548c36bd5dd125d84ae75ffca9015a14a2 ./sha3 -a sha3-384 -f sha3 sha3-384(sha3) = 7a0067b59facecbdd6f5fbd1a9b4fe64ae7cab49ed8657f2ac9c35fb4f8391e81cb84e4af01fc2a ./sha3 -a sha3-512 -f sha3 sha3-512(sha3) = bfb1ca400e2d63fadf49ec8f010d1e024655a09f47bed59208d73d99ae9fe6ece9cb4499b31e12f3f22aed12ceb5f06e39f226e5b9 ./sha3 -a shake128 -d 128 -f sha3 shake128(sha3) = 5be5b5fab03e8b08fcc0f39273efddab ./sha3 -a shake128 -f sha3 shake128(sha3) = 5be5b5fab03e8b08fcc0f39273efddab ./sha3 -a shake256 -d 256 -f sha3 shake256(sha3) = 542c1275f5ceeb0e4ceebb7b9d84b3b9cfcec575fdd57488bf8 ./sha3 -a shake256 -f sha3 shake256(sha3) = 542c1275f5ceeb0e4ceebb7b9d84b3b9cfcec575fdd57488bf8 

最新版本的openssl工具已经支持SHA3系列的哈希算法，因此可以将这里的sha3工具和openssl执行dgst计算的结果进行比较:

$ sha3 -h Usage: Common options: [-a sha3-224|sha3-256|sha3-384|sha3-512|shake128|shake256 [-d num]] [-x|-f file|-s string|-h] Hash a string: sha3 -a [sha3-224|sha3-256|sha3-384|sha3-512|shake128|shake256] [-d num] -s string Hash a file: sha3 -a [sha3-224|sha3-256|sha3-384|sha3-512|shake128|shake256] [-d num] -f file -a Secure hash algorithm: "sha3-224|sha3-256|sha3-384|sha3-512|shake128|shake256". Default: sha3-256 -d Digest length for shake128/shake256, required. Default: num=128[shake128], num=256[shake256] -x Internal string hash test -h Display this message # # 使用sha512工具分别对文件和字符串计算哈希值 # # 这里的字符串S="I Love China!"中有空格，所以在后面的处理中，需要用引号将字符串包含起来处理，否则会出现错误 $ S="I Love China!"; \ > for h in sha3-224 sha3-256 sha3-384 sha3-512 shake128 shake256; \ > do \ > echo "sha3 -a $h -s \"$S\""; \ > sha3 -a $h -s "$S"; \ > done; sha3 -a sha3-224 -s "I Love China!" sha3-224("I Love China!") = 473a00cbd6a244fa2a7e9c67bac0c1a09e4aadbefab00a156 sha3 -a sha3-256 -s "I Love China!" sha3-256("I Love China!") = 5f6416ad5e9bb92e994bf7d9c6a6769e6756e8dddb4b013e sha3 -a sha3-384 -s "I Love China!" sha3-384("I Love China!") = 1e0cfd7dc49d1a34a018b4def5b5fca68ac77803e494a6fd813abf116b6b720c0ad9ac477aa8c2c0 sha3 -a sha3-512 -s "I Love China!" sha3-512("I Love China!") = 8ebec4ced10b305c1a02c75c6c6dcf60ca5230b0a4befa4115d830df54ff049e7dbad25733ee7e8f8479dd654f906ef9b9fd44a4d94e5eeb76db sha3 -a shake128 -s "I Love China!" shake128("I Love China!") = 0e0fdd50977b9bdb0f0ab59f1 sha3 -a shake256 -s "I Love China!" shake256("I Love China!") = 0c3b1939e778be017330b9efe7ea7aae2a020ad7905fec88f66bec2 # # 使用开源的openssl工具计算相应的哈希进行对比 # $ S="I Love China!"; \ for h in sha3-224 sha3-256 sha3-384 sha3-512 shake128 shake256; \ do \ echo "echo -n \"$S\" | openssl dgst -$h"; \ echo -n "$S" | openssl dgst -$h; \ done; echo -n "I Love China!" | openssl dgst -sha3-224 (stdin)= 473a00cbd6a244fa2a7e9c67bac0c1a09e4aadbefab00a156 echo -n "I Love China!" | openssl dgst -sha3-256 (stdin)= 5f6416ad5e9bb92e994bf7d9c6a6769e6756e8dddb4b013e echo -n "I Love China!" | openssl dgst -sha3-384 (stdin)= 1e0cfd7dc49d1a34a018b4def5b5fca68ac77803e494a6fd813abf116b6b720c0ad9ac477aa8c2c0 echo -n "I Love China!" | openssl dgst -sha3-512 (stdin)= 8ebec4ced10b305c1a02c75c6c6dcf60ca5230b0a4befa4115d830df54ff049e7dbad25733ee7e8f8479dd654f906ef9b9fd44a4d94e5eeb76db echo -n "I Love China!" | openssl dgst -shake128 (stdin)= 0e0fdd50977b9bdb0f0ab59f1 echo -n "I Love China!" | openssl dgst -shake256 (stdin)= 0c3b1939e778be017330b9efe7ea7aae2a020ad7905fec88f66bec2 

完整代码

完整的代码文件列表如下:

sha3$ ls -lh total 68K -rwxr-xr-x 1 rg stb_all 1.2K Jun 24 11:07 Makefile -rwxr-xr-x 1 rg stb_all 15K Jun 24 14:47 sha3.c -rwxr-xr-x 1 rg stb_all 3.2K Jun 24 14:55 sha3.h -rwxr-xr-x 1 rg stb_all 26K Jun 24 15:05 sha3test.c -rwxr-xr-x 1 rg stb_all 758 Jun 23 17:33 utils.c -rwxr-xr-x 1 rg stb_all 1.8K Jun 21 09:48 utils.h 

需要代码请访问:

• https://github.com/guyongqiangx/cryptography/

其它

洛奇工作中常常会遇到自己不熟悉的问题，这些问题可能并不难，但因为不了解，找不到人帮忙而瞎折腾，往往导致浪费几天甚至更久的时间。

所以我组建了几个微信讨论群(记得微信我说加哪个群，如何加微信见后面)，欢迎一起讨论:

• 一个密码编码学讨论组，主要讨论各种加解密，签名校验等算法，请说明加密码学讨论群。
• 一个Android OTA的讨论组，请说明加Android OTA群。
• 一个git和repo的讨论组，请说明加git和repo群。

在工作之余，洛奇尽量写一些对大家有用的东西，如果洛奇的这篇文章让您有所收获，解决了您一直以来未能解决的问题，不妨赞赏一下洛奇，这也是对洛奇付出的最大鼓励。扫下面的二维码赞赏洛奇，金额随意：

洛奇自己维护了一个公众号“洛奇看世界”，一个很佛系的公众号，不定期瞎逼逼。公号也提供个人联系方式，一些资源，说不定会有意外的收获，详细内容见公号提示。扫下方二维码关注公众号：