rsa普密ukey

一、首先先看rsa的证书认证过程 #

1. tls1.2的rsa证书双向认证 Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d) #

1# 服务端命令
2openssl s_server -accept 7777 -state -debug -key domain.key -cert domain.crt -CAfile cacert.pem -Verify 1
3# 客户端命令
4openssl s_client -connect 127.0.0.1:7777 -cipher RSA -tls1_2 -debug -keylogfile test.log -cert domain.crt -key domain.key

@startuml tls

title tls1.2的rsa证书双向认证

participant client
participant server

== tcp建立完成后 ==

client->server: Client Hello
note over client
client hello
Content Type: Handshake(22)
Version: TLS 1.0 (0x0301)   这个可以忽略,Handshake默认就是tls1.0
Handshake Type: Client Hello(1)
    Version: TLS 1.2 (0x0303)  客户端支持的最高tls版本
    算法套件信息 Cipher Suites
        包含密钥交换算法、身份认证算法、对称加密算法、信息摘要算法
    随机数 Random
        前四个字节为GMT Unix时间,增强随机性
    请求的服务名 Extension: server_name(0)
    签名哈希算法列表 Extension: signature_algorithms(13)
end note

server->client: Server Hello, Certificate, Certificate Request, Server Hello Done
note over server
Server Hello
Content Type: Handshake(22)
Handshake Type: Server Hello(2)
    Version: TLS 1.2 (0x0303)  确定TLS版本为1.2
    Random 随机数
        前四个字节为GMT Unix时间,增强随机性
    Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d) 确定算法套件

end note
note over server
Certificate
Content Type: Handshake(22)
Handshake Type: Certificate(11)
    下发服务端证书
        包含证书信息、证书公钥
end note
note over server
Certificate
Content Type: Handshake(22)
Handshake Type: Certificate(11)
    下发服务端证书
        包含证书信息、证书公钥、证书签名信息
end note
note over server
Certificate Request
Content Type: Handshake(22)
Handshake Type: Certificate Request (13)
    要求客户端提交证书,一般是nginx配置中开启了对端证书校验
    Certificate types (3 types) 允许提交的证书类型
        RSA Sign
        DSS Sign
        ECDSA Sign
    Signature Hash Algorithms (23 algorithms) 支持的签名和hash算法
    Distinguished Names (135 bytes) CA相关信息,只支持此CA下发的相关证书
end note
note over server
Server Hello Done
Content Type: Handshake(22)
Handshake Type: Server Hello Done(14)
server hello结束
end note

client->server: Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message
note over client
Certificate
Content Type: Handshake(22)
Handshake Type: Certificate(11)
    提交客户端证书
    Certificates (2043 bytes) 证书信息集
        Certificate: 单个证书信息
            signedCertificate 证书的基本信息
                subject 主题
                subjectPublicKeyInfo 公钥信息
                validity 证书有效期
            algorithmIdentifier (sha256WithRSAEncryption)
            encrypted: 证书签名值(每个证书都自带(签发时使用私钥加密摘要信息算出来的),用于防篡改,不是tls过程中算出来的)
end note
note over client
Client Key Exchange
Content Type: Handshake(22)
Handshake Type: Client Key Exchange(16)
    生成预主密钥,使用服务端下发的公钥加密上传到服务端,由于只有服务端自己有私钥这一步保证证书是服务端自己持有的
end note
note over client
Certificate Verify
Content Type: Handshake(22)
Handshake Type: Certificate Verify (15)
    使用客户端证书的私钥对握手信息进行签名,由服务端进行验签来确定证书是客户端自己持有的
end note
note over client
Change Chipher Spec
Content Type: Change Chipher Spec(20)
    告知服务端,下面的数据要加密了
end note
note over client
Finished
Opaque Type: Application Data(23)
    下面是解密后的数据
    Content Type: Handshake(22)
    Handshake Type: Finished(20)
end note

server->client: New Session Ticket, Change Cipher Spec, Finished
note over server
New Session Ticket
Content Type: Change Chipher Spec(20)
Handshake Type: New Session Ticket (4)
    给一个session到客户端,用于ssl复用
end note
note over server
Change Chipher Spec
Content Type: Change Chipher Spec(20)
    告知客户端,下面的数据要加密了
end note
note over server
Finished
Opaque Type: Application Data(23)
    下面是解密后的数据
    Content Type: Handshake(22)
    Handshake Type: Finished(20)
end note

== 开始ssl数据传输 ==

@enduml

从上面的过程中,我们知道有几个关键点

  1. 证书主体内容作为Certificate发出去
  2. Client Verify需要使用证书的私钥对握手信息进行签名

二、如何开发一个引擎 #

1. 证书双向认证引擎 #

  • 从上面的流程可以得出,证书认证中需要使用客户端证书的部分只有CertificateCertificate Verify两个请求
  • 一般使用ukey进行双向认证的话,私钥是无法导出的,所以需要实现一下证书导出和私钥签名函数

1.1. 引擎基础设置 #

 1/******************** 这个函数是引擎设置的最基础的函数 ********************/
 2/* Constants used when creating the ENGINE */
 3static const char* engine_id = "skf";
 4static const char* engine_name = "skf hardware engine support";
 5static RSA_METHOD* skf_rsa_method = nullptr;
 6/* Prepare the ENGINE structure for registration */
 7static int skf_bind_helper(ENGINE* e) {
 8    LOG_DEBUG("skf_bind_helper");
 9
10    // set skf rsa engine method, use default method and diy some function
11    if ((skf_rsa_method = RSA_meth_dup(RSA_get_default_method())) == NULL ||
12        // 私钥加密函数
13        RSA_meth_set_priv_enc(skf_rsa_method, skf_rsa_priv_enc) == 0) {
14        return 0;
15    }
16
17    if (!ENGINE_set_id(e, engine_id) ||                     // 引擎id
18        !ENGINE_set_name(e, engine_name) ||                 // 引擎名字
19        !ENGINE_set_init_function(e, engine_init) ||        // 初始化函数,握手前调用
20        !ENGINE_set_finish_function(e, engine_finish) ||    // 结束函数,连接完成不再需要引擎时调用
21        !ENGINE_set_destroy_function(e, engine_destroy) ||  // 销毁函数,结束时调用,只会被调用一次
22        !ENGINE_set_RSA(e, skf_rsa_method) ||               // 设置rsa算法结构体
23        // if not set, sdf will load by default if possible
24        !ENGINE_set_flags(e, ENGINE_FLAGS_NO_REGISTER_ALL) ||
25        !ENGINE_set_load_ssl_client_cert_function(e, skf_load_ssl_client_cert)  // 设置客户端证书加载函数
26    ) {
27        return 0;
28    }
29
30    return 1;
31}
32
33/* This stuff is needed if this ENGINE is being compiled into a self-contained
34 * shared-library. */
35#include <openssl/engine.h>
36#ifndef OPENSSL_NO_DYNAMIC_ENGINE
37
38extern "C" {
39static int bind_helper(ENGINE* e, const char* id) {
40    if (id && (strcmp(id, engine_id) != 0)) return 0;
41
42    if (!skf_bind_helper(e)) return 0;
43
44    return 1;
45}
46
47IMPLEMENT_DYNAMIC_CHECK_FN()
48IMPLEMENT_DYNAMIC_BIND_FN(bind_helper)
49}
50#else
51static ENGINE* engine_skf(void) {
52    ENGINE* eng = ENGINE_new();
53    if (!eng) return NULL;
54    if (!skf_bind_helper(eng)) {
55        ENGINE_free(eng);
56        return NULL;
57    }
58
59    return eng;
60}
61
62void engine_load_skf_int(void) {
63    ENGINE* toadd = engine_skf();
64    if (!toadd) return;
65    ENGINE_add(toadd);
66    ENGINE_free(toadd);
67    ERR_clear_error();
68}
69#endif /* OPENSSL_NO_DYNAMIC_ENGINE */

1.2. 导出证书的函数 #

  • 注册证书的加载函数到openssl
 1static int skf_bind_helper(ENGINE* e) {
 2    ...
 3    if (...
 4        !ENGINE_set_load_ssl_client_cert_function(e, skf_load_ssl_client_cert)  // 设置客户端证书加载函数
 5    ) {
 6        return 0;
 7    }
 8
 9    return 1;
10}
  • 函数原型为
1static int skf_load_ssl_client_cert(ENGINE* e, SSL* ssl, STACK_OF(X509_NAME) * ca_dn, X509** pcert, EVP_PKEY** ppkey,
2                                    STACK_OF(X509) * *pother, UI_METHOD* ui_method, void* callback_data);
  • 一个简单的软件实现代码如下,从文件中读取证书给到引擎
  • 使用ukey的话,这里就只能到处证书和公钥,私钥无法导出
 1/**
 2 * @brief 导出客户端证书函数
 3 *
 4 * @param[in] e 引擎指针
 5 * @param[in] ssl ssl结构体
 6 * @param[in] ca_dn ca相关信息,一般是服务端下发的用于匹配证书
 7 * @param[out] pcert 证书指针,内部申请空间,外部会进行释放
 8 * @param[out] ppkey 算法key结构体,里面其实是只需要公钥的
 9 * @param[out] pother
10 * @param[in] ui_method
11 * @param[in] callback_data
12 * @return int 1成功,0失败
13 */
14static int skf_load_ssl_client_cert(ENGINE* e, SSL* ssl, STACK_OF(X509_NAME) * ca_dn, X509** pcert, EVP_PKEY** ppkey,
15                                    STACK_OF(X509) * *pother, UI_METHOD* ui_method, void* callback_data) {
16    const char* operation = "skf_load_ssl_client_cert";
17    LOG_DEBUG("%s", operation);
18    int iResult = 0;
19    // 读取PEM格式证书文件
20    FILE* fp = fopen("/path/to/domain.crt", "r");
21    *pcert = PEM_read_X509(fp, NULL, NULL, NULL);
22    fclose(fp);
23    if (*pcert == NULL) {
24        LOG_ERROR("[%s], PEM_read_X509 failed", operation);
25        return 0;
26    }
27    // 直接从证书读取公钥放这里最省事,外面会进行校验公钥和证书是否匹配
28    // ukey只能导出公钥,如果是其他可以导出私钥的,这里也可以将私钥放进去,这样可以直接软件实现私钥加密函数
29    // 这里要增加引用计数,对外部来说,ppkey和pcert是分开释放的
30    *ppkey = X509_get_pubkey(*pcert);
31    LOG_INFO("%s success", operation);
32    return 1;
33}

3. 私钥签名 #

  • ukey中的私钥不可以导出,所以要注册私钥签名函数到openssl中
 1/* Prepare the ENGINE structure for registration */
 2static int skf_bind_helper(ENGINE* e) {
 3    LOG_DEBUG(Tag, "skf_bind_helper");
 4
 5    // set skf rsa engine method, use default method and diy some function
 6    if ((skf_rsa_method = RSA_meth_dup(RSA_get_default_method())) == NULL ||
 7        // 提供私钥加密函数
 8        RSA_meth_set_priv_enc(skf_rsa_method, skf_rsa_priv_enc) == 0) {
 9        return 0;
10    }
11    ...
12    return 1;
13}
  • 这里实现一个从文件读出私钥然后调用默认的openssl的rsa私钥签名函数
 1int skf_rsa_priv_enc(int flen, const unsigned char* from, unsigned char* to, RSA* rsa, int padding) {
 2    LOG_DEBUG("skf_rsa_priv_enc from len %d, padding %d", flen, padding);
 3
 4    // 读取PEM格式私钥文件
 5    FILE* fp = fopen("/home/wangyubo/work/src/local/openssl/20230418/ssl_diy/domain.key", "r");
 6    auto evkey = PEM_read_PrivateKey(fp, NULL, NULL, NULL);
 7    fclose(fp);
 8    if (evkey == NULL) {
 9        LOG_ERROR("d2i_PrivateKey_fp failed");
10        return 0;
11    }
12
13    auto newRsa = EVP_PKEY_get0_RSA(evkey);
14    int ret = RSA_meth_get_priv_enc(RSA_get_default_method())(flen, from, to, newRsa, padding);
15
16    LOG_DEBUG("rsaDoSign ret %d", ret);
17    return ret;
18}

三、握手过程的调用栈 #

1. Certificate中证书信息的获取 #

1.1. 在收到服务端下发的Certificate Request的时候,就先加载证书到ssl的上下文中 #

 1WORK_STATE tls_prepare_client_certificate(SSL *s, WORK_STATE wst)
 2{
 3    X509 *x509 = NULL;
 4    EVP_PKEY *pkey = NULL;
 5    int i;
 6
 7    if (wst == WORK_MORE_A) {
 8        // 这里一般是证书已经通过命令行设置好了才进来的
 9        /* Let cert callback update client certificates if required */
10        if (s->cert->cert_cb) {
11            i = s->cert->cert_cb(s, s->cert->cert_cb_arg);
12            if (i < 0) {
13                s->rwstate = SSL_X509_LOOKUP;
14                return WORK_MORE_A;
15            }
16            if (i == 0) {
17                SSLfatal(s, SSL_AD_INTERNAL_ERROR, SSL_R_CALLBACK_FAILED);
18                return WORK_ERROR;
19            }
20            s->rwstate = SSL_NOTHING;
21        }
22        if (ssl3_check_client_certificate(s)) {
23            if (s->post_handshake_auth == SSL_PHA_REQUESTED) {
24                return WORK_FINISHED_STOP;
25            }
26            return WORK_FINISHED_CONTINUE;
27        }
28
29        /* Fall through to WORK_MORE_B */
30        wst = WORK_MORE_B;
31    }
32
33    /* We need to get a client cert */
34    if (wst == WORK_MORE_B) {
35        // 通过引擎设置证书会走到这个逻辑
36        /*
37         * If we get an error, we need to ssl->rwstate=SSL_X509_LOOKUP;
38         * return(-1); We then get retied later
39         */
40        // 这里调用到引擎的处理函数中
41        i = ssl_do_client_cert_cb(s, &x509, &pkey);
42        if (i < 0) {
43            s->rwstate = SSL_X509_LOOKUP;
44            return WORK_MORE_B;
45        }
46        s->rwstate = SSL_NOTHING;
47        if ((i == 1) && (pkey != NULL) && (x509 != NULL)) {
48            // 证书设置到ssl结构体
49            // 验证pkey和x509是否匹配,pkey中只需要公钥匹配即可
50            if (!SSL_use_certificate(s, x509) || !SSL_use_PrivateKey(s, pkey))
51                i = 0;
52        } else if (i == 1) {
53            i = 0;
54            ERR_raise(ERR_LIB_SSL, SSL_R_BAD_DATA_RETURNED_BY_CALLBACK);
55        }
56
57        X509_free(x509);
58        EVP_PKEY_free(pkey);
59        if (i && !ssl3_check_client_certificate(s))
60            i = 0;
61        if (i == 0) {
62            if (s->version == SSL3_VERSION) {
63                s->s3.tmp.cert_req = 0;
64                ssl3_send_alert(s, SSL3_AL_WARNING, SSL_AD_NO_CERTIFICATE);
65                return WORK_FINISHED_CONTINUE;
66            } else {
67                s->s3.tmp.cert_req = 2;
68                if (!ssl3_digest_cached_records(s, 0)) {
69                    /* SSLfatal() already called */
70                    return WORK_ERROR;
71                }
72            }
73        }
74
75        if (s->post_handshake_auth == SSL_PHA_REQUESTED)
76            return WORK_FINISHED_STOP;
77        return WORK_FINISHED_CONTINUE;
78    }
79
80    /* Shouldn't ever get here */
81    SSLfatal(s, SSL_AD_INTERNAL_ERROR, ERR_R_INTERNAL_ERROR);
82    return WORK_ERROR;
83}

1) 从引擎获取证书 #

 1/*
 2libssl.so.3!tls_engine_load_ssl_client_cert(SSL *s, X509 **px509, EVP_PKEY **ppkey) (ssl/tls_depr.c:69)
 3libssl.so.3!ssl_do_client_cert_cb(SSL *s, X509 **px509, EVP_PKEY **ppkey) (ssl/statem/statem_clnt.c:3663)
 4libssl.so.3!tls_prepare_client_certificate(SSL * s, WORK_STATE wst) (ssl/statem/statem_clnt.c:3460)
 5libssl.so.3!ossl_statem_client_post_process_message(SSL * s, WORK_STATE wst) (ssl/statem/statem_clnt.c:1089)
 6libssl.so.3!read_state_machine(SSL * s) (ssl/statem/statem.c:675)
 7libssl.so.3!state_machine(SSL * s, int server) (ssl/statem/statem.c:442)
 8libssl.so.3!ossl_statem_connect(SSL * s) (ssl/statem/statem.c:265)
 9libssl.so.3!ssl3_write_bytes(SSL * s, int type, const void * buf_, size_t len, size_t * written) (ssl/record/rec_layer_s3.c:398)
10libssl.so.3!ssl3_write(SSL * s, const void * buf, size_t len, size_t * written) (ssl/s3_lib.c:4449)
11libssl.so.3!ssl_write_internal(SSL * s, const void * buf, size_t num, size_t * written) (ssl/ssl_lib.c:2062)
12libssl.so.3!SSL_write(SSL * s, const void * buf, int num) (ssl/ssl_lib.c:2140)
13s_client_main(int argc, char ** argv) (apps/s_client.c:2841)
14do_cmd(struct lhash_st_FUNCTION * prog, int argc, char ** argv) (apps/openssl.c:418)
15main(int argc, char ** argv) (apps/openssl.c:298)
16 */
17#ifndef OPENSSL_NO_ENGINE
18int tls_engine_load_ssl_client_cert(SSL *s, X509 **px509, EVP_PKEY **ppkey)
19{
20    return ENGINE_load_ssl_client_cert(s->ctx->client_cert_engine, s,
21                                       SSL_get_client_CA_list(s),
22                                       px509, ppkey, NULL, NULL, NULL);
23}
24#endif

2) 随着证书到处的key中为什么只需要公钥即可 #

  • pkey中为什么只需要公钥即可,看下面的调用栈,从SSL_use_PrivateKey进来
  • 检查ppkey就是使用证书的公钥去匹配
  • 本身证书中也只有公钥,想要看证书和ppkey是否匹配,就是看公钥的类型、算法、数值是否一致
 1/*
 2libcrypto.so.3!X509_check_private_key(const X509 * x, const EVP_PKEY * k) (crypto/x509/x509_cmp.c:403)
 3libssl.so.3!ssl_set_pkey(CERT * c, EVP_PKEY * pkey) (ssl/ssl_rsa.c:128)
 4libssl.so.3!SSL_use_PrivateKey(SSL * ssl, EVP_PKEY * pkey) (ssl/ssl_rsa.c:146)
 5libssl.so.3!tls_prepare_client_certificate(SSL * s, WORK_STATE wst) (ssl/statem/statem_clnt.c:3467)
 6libssl.so.3!ossl_statem_client_post_process_message(SSL * s, WORK_STATE wst) (ssl/statem/statem_clnt.c:1089)
 7libssl.so.3!read_state_machine(SSL * s) (ssl/statem/statem.c:675)
 8libssl.so.3!state_machine(SSL * s, int server) (ssl/statem/statem.c:442)
 9libssl.so.3!ossl_statem_connect(SSL * s) (ssl/statem/statem.c:265)
10libssl.so.3!ssl3_write_bytes(SSL * s, int type, const void * buf_, size_t len, size_t * written) (ssl/record/rec_layer_s3.c:398)
11libssl.so.3!ssl3_write(SSL * s, const void * buf, size_t len, size_t * written) (ssl/s3_lib.c:4449)
12libssl.so.3!ssl_write_internal(SSL * s, const void * buf, size_t num, size_t * written) (ssl/ssl_lib.c:2062)
13libssl.so.3!SSL_write(SSL * s, const void * buf, int num) (ssl/ssl_lib.c:2140)
14s_client_main(int argc, char ** argv) (apps/s_client.c:2841)
15do_cmd(struct lhash_st_FUNCTION * prog, int argc, char ** argv) (apps/openssl.c:418)
16main(int argc, char ** argv) (apps/openssl.c:298)
17 */
18int X509_check_private_key(const X509 *x, const EVP_PKEY *k)
19{
20    const EVP_PKEY *xk;
21    int ret;
22
23    xk = X509_get0_pubkey(x);
24    if (xk == NULL) {
25        ERR_raise(ERR_LIB_X509, X509_R_UNABLE_TO_GET_CERTS_PUBLIC_KEY);
26        return 0;
27    }
28
29    switch (ret = EVP_PKEY_eq(xk, k)) {
30    case 0:
31        ERR_raise(ERR_LIB_X509, X509_R_KEY_VALUES_MISMATCH);
32        break;
33    case -1:
34        ERR_raise(ERR_LIB_X509, X509_R_KEY_TYPE_MISMATCH);
35        break;
36    case -2:
37        ERR_raise(ERR_LIB_X509, X509_R_UNKNOWN_KEY_TYPE);
38        break;
39    }
40
41    return ret > 0;
42}

1.2. 构造Certificate请求时,把证书信息放到包里面 #

 1/*
 2libssl.so.3!tls_construct_client_certificate(SSL * s, WPACKET * pkt) (ssl/statem/statem_clnt.c:3504)
 3libssl.so.3!write_state_machine(SSL * s) (ssl/statem/statem.c:855)
 4libssl.so.3!state_machine(SSL * s, int server) (ssl/statem/statem.c:451)
 5libssl.so.3!ossl_statem_connect(SSL * s) (ssl/statem/statem.c:265)
 6libssl.so.3!ssl3_write_bytes(SSL * s, int type, const void * buf_, size_t len, size_t * written) (ssl/record/rec_layer_s3.c:398)
 7libssl.so.3!ssl3_write(SSL * s, const void * buf, size_t len, size_t * written) (ssl/s3_lib.c:4449)
 8libssl.so.3!ssl_write_internal(SSL * s, const void * buf, size_t num, size_t * written) (ssl/ssl_lib.c:2062)
 9libssl.so.3!SSL_write(SSL * s, const void * buf, int num) (ssl/ssl_lib.c:2140)
10s_client_main(int argc, char ** argv) (apps/s_client.c:2841)
11do_cmd(struct lhash_st_FUNCTION * prog, int argc, char ** argv) (apps/openssl.c:418)
12main(int argc, char ** argv) (apps/openssl.c:298)
13 */
14int tls_construct_client_certificate(SSL *s, WPACKET *pkt)
15{
16    if (SSL_IS_TLS13(s)) {
17        if (s->pha_context == NULL) {
18            /* no context available, add 0-length context */
19            if (!WPACKET_put_bytes_u8(pkt, 0)) {
20                SSLfatal(s, SSL_AD_INTERNAL_ERROR, ERR_R_INTERNAL_ERROR);
21                return 0;
22            }
23        } else if (!WPACKET_sub_memcpy_u8(pkt, s->pha_context, s->pha_context_len)) {
24            SSLfatal(s, SSL_AD_INTERNAL_ERROR, ERR_R_INTERNAL_ERROR);
25            return 0;
26        }
27    }
28    if (!ssl3_output_cert_chain(s, pkt,
29                                (s->s3.tmp.cert_req == 2) ? NULL
30                                                           : s->cert->key)) {
31        /* SSLfatal() already called */
32        return 0;
33    }
34
35    if (SSL_IS_TLS13(s)
36            && SSL_IS_FIRST_HANDSHAKE(s)
37            && (!s->method->ssl3_enc->change_cipher_state(s,
38                    SSL3_CC_HANDSHAKE | SSL3_CHANGE_CIPHER_CLIENT_WRITE))) {
39        /*
40         * This is a fatal error, which leaves enc_write_ctx in an inconsistent
41         * state and thus ssl3_send_alert may crash.
42         */
43        SSLfatal(s, SSL_AD_NO_ALERT, SSL_R_CANNOT_CHANGE_CIPHER);
44        return 0;
45    }
46
47    return 1;
48}

2. certificate verify中的签名信息的生成 #

  • 签名信息是将一段信息使用rsa私钥签名后的结果,先算hash后签名
  • 这里的rsa_sign进来的数据已经做了hash,只需要进行签名即可
  1/*
  2libcrypto.so.3!rsa_sign(void * vprsactx, unsigned char * sig, size_t * siglen, size_t sigsize, const unsigned char * tbs, size_t tbslen) (providers/implementations/signature/rsa_sig.c:647)
  3libcrypto.so.3!rsa_digest_sign_final(void * vprsactx, unsigned char * sig, size_t * siglen, size_t sigsize) (providers/implementations/signature/rsa_sig.c:942)
  4libcrypto.so.3!EVP_DigestSignFinal(EVP_MD_CTX * ctx, unsigned char * sigret, size_t * siglen) (crypto/evp/m_sigver.c:481)
  5libcrypto.so.3!EVP_DigestSign(EVP_MD_CTX * ctx, unsigned char * sigret, size_t * siglen, const unsigned char * tbs, size_t tbslen) (crypto/evp/m_sigver.c:582)
  6libssl.so.3!tls_construct_cert_verify(SSL * s, WPACKET * pkt) (ssl/statem/statem_lib.c:354)
  7libssl.so.3!write_state_machine(SSL * s) (ssl/statem/statem.c:855)
  8libssl.so.3!state_machine(SSL * s, int server) (ssl/statem/statem.c:451)
  9libssl.so.3!ossl_statem_connect(SSL * s) (ssl/statem/statem.c:265)
 10libssl.so.3!ssl3_write_bytes(SSL * s, int type, const void * buf_, size_t len, size_t * written) (ssl/record/rec_layer_s3.c:398)
 11libssl.so.3!ssl3_write(SSL * s, const void * buf, size_t len, size_t * written) (ssl/s3_lib.c:4449)
 12libssl.so.3!ssl_write_internal(SSL * s, const void * buf, size_t num, size_t * written) (ssl/ssl_lib.c:2062)
 13libssl.so.3!SSL_write(SSL * s, const void * buf, int num) (ssl/ssl_lib.c:2140)
 14s_client_main(int argc, char ** argv) (apps/s_client.c:2841)
 15do_cmd(struct lhash_st_FUNCTION * prog, int argc, char ** argv) (apps/openssl.c:418)
 16main(int argc, char ** argv) (apps/openssl.c:298)
 17 */
 18static int rsa_sign(void *vprsactx, unsigned char *sig, size_t *siglen,
 19                    size_t sigsize, const unsigned char *tbs, size_t tbslen)
 20{
 21    PROV_RSA_CTX *prsactx = (PROV_RSA_CTX *)vprsactx;
 22    int ret;
 23    size_t rsasize = RSA_size(prsactx->rsa);
 24    size_t mdsize = rsa_get_md_size(prsactx);
 25
 26    if (!ossl_prov_is_running())
 27        return 0;
 28
 29    if (sig == NULL) {
 30        *siglen = rsasize;
 31        return 1;
 32    }
 33
 34    if (sigsize < rsasize) {
 35        ERR_raise_data(ERR_LIB_PROV, PROV_R_INVALID_SIGNATURE_SIZE,
 36                       "is %zu, should be at least %zu", sigsize, rsasize);
 37        return 0;
 38    }
 39
 40    if (mdsize != 0) {
 41        if (tbslen != mdsize) {
 42            ERR_raise(ERR_LIB_PROV, PROV_R_INVALID_DIGEST_LENGTH);
 43            return 0;
 44        }
 45
 46#ifndef FIPS_MODULE
 47        if (EVP_MD_is_a(prsactx->md, OSSL_DIGEST_NAME_MDC2)) {
 48            unsigned int sltmp;
 49
 50            if (prsactx->pad_mode != RSA_PKCS1_PADDING) {
 51                ERR_raise_data(ERR_LIB_PROV, PROV_R_INVALID_PADDING_MODE,
 52                               "only PKCS#1 padding supported with MDC2");
 53                return 0;
 54            }
 55            ret = RSA_sign_ASN1_OCTET_STRING(0, tbs, tbslen, sig, &sltmp,
 56                                             prsactx->rsa);
 57
 58            if (ret <= 0) {
 59                ERR_raise(ERR_LIB_PROV, ERR_R_RSA_LIB);
 60                return 0;
 61            }
 62            ret = sltmp;
 63            goto end;
 64        }
 65#endif
 66
 67        // 根据不同的padding算法进行不同的计算
 68        switch (prsactx->pad_mode) {
 69        case RSA_X931_PADDING:
 70            if ((size_t)RSA_size(prsactx->rsa) < tbslen + 1) {
 71                ERR_raise_data(ERR_LIB_PROV, PROV_R_KEY_SIZE_TOO_SMALL,
 72                               "RSA key size = %d, expected minimum = %d",
 73                               RSA_size(prsactx->rsa), tbslen + 1);
 74                return 0;
 75            }
 76            if (!setup_tbuf(prsactx)) {
 77                ERR_raise(ERR_LIB_PROV, ERR_R_MALLOC_FAILURE);
 78                return 0;
 79            }
 80            memcpy(prsactx->tbuf, tbs, tbslen);
 81            prsactx->tbuf[tbslen] = RSA_X931_hash_id(prsactx->mdnid);
 82            ret = RSA_private_encrypt(tbslen + 1, prsactx->tbuf,
 83                                      sig, prsactx->rsa, RSA_X931_PADDING);
 84            clean_tbuf(prsactx);
 85            break;
 86
 87        case RSA_PKCS1_PADDING:
 88            {
 89                unsigned int sltmp;
 90
 91                ret = RSA_sign(prsactx->mdnid, tbs, tbslen, sig, &sltmp,
 92                               prsactx->rsa);
 93                if (ret <= 0) {
 94                    ERR_raise(ERR_LIB_PROV, ERR_R_RSA_LIB);
 95                    return 0;
 96                }
 97                ret = sltmp;
 98            }
 99            break;
100
101        case RSA_PKCS1_PSS_PADDING:
102            /* Check PSS restrictions */
103            if (rsa_pss_restricted(prsactx)) {
104                switch (prsactx->saltlen) {
105                case RSA_PSS_SALTLEN_DIGEST:
106                    if (prsactx->min_saltlen > EVP_MD_get_size(prsactx->md)) {
107                        ERR_raise_data(ERR_LIB_PROV,
108                                       PROV_R_PSS_SALTLEN_TOO_SMALL,
109                                       "minimum salt length set to %d, "
110                                       "but the digest only gives %d",
111                                       prsactx->min_saltlen,
112                                       EVP_MD_get_size(prsactx->md));
113                        return 0;
114                    }
115                    /* FALLTHRU */
116                default:
117                    if (prsactx->saltlen >= 0
118                        && prsactx->saltlen < prsactx->min_saltlen) {
119                        ERR_raise_data(ERR_LIB_PROV,
120                                       PROV_R_PSS_SALTLEN_TOO_SMALL,
121                                       "minimum salt length set to %d, but the"
122                                       "actual salt length is only set to %d",
123                                       prsactx->min_saltlen,
124                                       prsactx->saltlen);
125                        return 0;
126                    }
127                    break;
128                }
129            }
130            if (!setup_tbuf(prsactx))
131                return 0;
132            if (!RSA_padding_add_PKCS1_PSS_mgf1(prsactx->rsa,
133                                                prsactx->tbuf, tbs,
134                                                prsactx->md, prsactx->mgf1_md,
135                                                prsactx->saltlen)) {
136                ERR_raise(ERR_LIB_PROV, ERR_R_RSA_LIB);
137                return 0;
138            }
139            // 一般tls1.2的rsa证书双向认证会调用到这个私钥签名函数
140            ret = RSA_private_encrypt(RSA_size(prsactx->rsa), prsactx->tbuf,
141                                      sig, prsactx->rsa, RSA_NO_PADDING);
142            clean_tbuf(prsactx);
143            break;
144
145        default:
146            ERR_raise_data(ERR_LIB_PROV, PROV_R_INVALID_PADDING_MODE,
147                           "Only X.931, PKCS#1 v1.5 or PSS padding allowed");
148            return 0;
149        }
150    } else {
151        ret = RSA_private_encrypt(tbslen, tbs, sig, prsactx->rsa,
152                                  prsactx->pad_mode);
153    }
154
155#ifndef FIPS_MODULE
156 end:
157#endif
158    if (ret <= 0) {
159        ERR_raise(ERR_LIB_PROV, ERR_R_RSA_LIB);
160        return 0;
161    }
162
163    *siglen = ret;
164    return 1;
165}
  • 私钥签名函数会调用到引擎里面的rsa_priv_enc函数,也就是上面注册的函数
1int RSA_private_encrypt(int flen, const unsigned char *from,
2                        unsigned char *to, RSA *rsa, int padding)
3{
4    return rsa->meth->rsa_priv_enc(flen, from, to, rsa, padding);
5}