对private_key使用shared_ptr时出现分段错误

Question

更新

[X] 当TLS::credentials creds在上声明时，我发现了这种情况全局范围，但如果我在seg之外声明它，故障就不会发生。

我需要它是全局的，因为它有助于缓存证书和多个线程可以使用其他线程创建的证书花费时间创建新证书。

[X] 我进一步将代码从大约200行减少到100行

我正在使用Botan创建TLS应用程序，我的应用程序在应用程序结束时出现seg故障而崩溃。

我试着用Valgrind调试这个，但没有结果。

这是来自Valgrind，的堆栈轨迹

==3841967== Invalid write of size 8
==3841967==    at 0x4842964: memset (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3841967==    by 0x566A82F: Botan::deallocate_memory(void*, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x55E1A4D: ??? (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==    by 0x40EC29: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:730)
==3841967==    by 0x41112D: std::__shared_ptr<Botan::RSA_Public_Data const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1169)
==3841967==    by 0x411107: std::shared_ptr<Botan::RSA_Public_Data const>::~shared_ptr() (shared_ptr.h:103)
==3841967==    by 0x41109D: Botan::RSA_PublicKey::~RSA_PublicKey() (rsa.h:25)
==3841967==    by 0x410FC1: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410DC5: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410E8A: std::_Sp_counted_ptr<Botan::RSA_PrivateKey*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==  Address 0x9419080 is not stack'd, malloc'd or (recently) free'd
==3841967== 
==3841967== 
==3841967== Process terminating with default action of signal 11 (SIGSEGV)
==3841967==  Access not within mapped region at address 0x9419080
==3841967==    at 0x4842964: memset (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3841967==    by 0x566A82F: Botan::deallocate_memory(void*, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x55E1A4D: ??? (in /usr/lib/x86_64-linux-gnu/libbotan-2.so.12.12.1)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==    by 0x40EC29: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:730)
==3841967==    by 0x41112D: std::__shared_ptr<Botan::RSA_Public_Data const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1169)
==3841967==    by 0x411107: std::shared_ptr<Botan::RSA_Public_Data const>::~shared_ptr() (shared_ptr.h:103)
==3841967==    by 0x41109D: Botan::RSA_PublicKey::~RSA_PublicKey() (rsa.h:25)
==3841967==    by 0x410FC1: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410DC5: Botan::RSA_PrivateKey::~RSA_PrivateKey() (rsa.h:92)
==3841967==    by 0x410E8A: std::_Sp_counted_ptr<Botan::RSA_PrivateKey*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
==3841967==    by 0x40EC7B: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==3841967==  If you believe this happened as a result of a stack
==3841967==  overflow in your program's main thread (unlikely but
==3841967==  possible), you can try to increase the size of the
==3841967==  main thread stack using the --main-stacksize= flag.
==3841967==  The main thread stack size used in this run was 8388608.
==3841967== 
==3841967== HEAP SUMMARY:
==3841967==     in use at exit: 149,626 bytes in 1,143 blocks
==3841967==   total heap usage: 211,782 allocs, 210,639 frees, 90,582,963 bytes allocated
==3841967== 
==3841967== LEAK SUMMARY:
==3841967==    definitely lost: 0 bytes in 0 blocks
==3841967==    indirectly lost: 0 bytes in 0 blocks
==3841967==      possibly lost: 1,352 bytes in 18 blocks
==3841967==    still reachable: 148,274 bytes in 1,125 blocks
==3841967==                       of which reachable via heuristic:
==3841967==                         newarray           : 1,536 bytes in 16 blocks
==3841967==         suppressed: 0 bytes in 0 blocks
==3841967== Rerun with --leak-check=full to see details of leaked memory
==3841967== 
==3841967== For lists of detected and suppressed errors, rerun with: -s
==3841967== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

你可以通过发布将植物克隆到你的机器中

git clone https://github.com/randombit/botan.git

然后按照他们官方网站上的说明来构建&安装它。

您需要创建一个根证书颁发机构来与应用程序一起使用，为此，您必须在计算机上安装OpenSSL。

在其中创建一个名为testApplication和cd的文件夹。

然后使用Bash，发出以下一系列命令来创建根CA，

# Generate private key
openssl genrsa -des3 -out myCA.key 2048
# Generate root certificate
openssl req -x509 -new -nodes -key myCA.key -sha256 -days 825 -out myCA.pem
# Convert to Botan Format
openssl pkcs8 -topk8 -in myCA.key > myCAKey.pkcs8.pem

请使用thisispassword作为密码。

在你的机器上安装clang编译器，然后你就可以编译如下的源文件，

clang++ example.cpp -o example  -Wthread-safety -Wall -Wextra -g -std=c++17 -pthread -lssl -lcrypto -lbotan-2 --I/usr/include/botan-2

example.cpp

#include <iostream>
#include <string>
#include <vector>
#include <fstream>
#include <sstream>
#include <botan/tls_server.h>
#include <botan/tls_callbacks.h>
#include <botan/tls_session_manager.h>
#include <botan/tls_policy.h>
#include <botan/auto_rng.h>
#include <botan/certstor.h>
#include <botan/pk_keys.h>
#include <botan/pkcs10.h>
#include <botan/pkcs8.h>
#include <botan/x509self.h>
#include <botan/x509path.h>
#include <botan/x509_ca.h>
#include <botan/x509_ext.h>
#include <botan/pk_algs.h>
#include <botan/ber_dec.h>
#include <botan/der_enc.h>
#include <botan/oids.h>
#include <botan/rsa.h>
namespace TLS
{
typedef std::chrono::duration<int, std::ratio<31556926>> years;
class credentials : public Botan::Credentials_Manager
{
private:
struct certificate
{
std::vector<Botan::X509_Certificate> certs;
std::shared_ptr<Botan::Private_Key> key;
};
std::vector<certificate> creds;
std::vector<std::shared_ptr<Botan::Certificate_Store>> store;
public:
void createCert(std::string hostname)
{
/**
* Initialize Root CA
**/
Botan::AutoSeeded_RNG rng;
const Botan::X509_Certificate rootCert("myCA.pem");
std::ifstream rootCertPrivateKeyFile("myCAKey.pkcs8.pem");
Botan::DataSource_Stream rootCertPrivateKeyStream(rootCertPrivateKeyFile);
std::unique_ptr<Botan::Private_Key> rootCertPrivateKey = Botan::PKCS8::load_key(rootCertPrivateKeyStream, "thisispassword");
Botan::X509_CA rootCA(rootCert, *rootCertPrivateKey, "SHA-256", rng);
/**
* Generate a Cert & Sign with Root CA
**/
Botan::X509_Cert_Options opts;
std::shared_ptr<Botan::Private_Key> serverPrivateKeyShared(new Botan::RSA_PrivateKey(rng, 4096));
Botan::RSA_PrivateKey* serverPrivateKey = (Botan::RSA_PrivateKey*)serverPrivateKeyShared.get();
opts.common_name = hostname;
opts.country = "US";
auto now = std::chrono::system_clock::now();
Botan::X509_Time todayDate(now);
Botan::X509_Time expireDate(now + years(1));
Botan::PKCS10_Request req = Botan::X509::create_cert_req(opts, *serverPrivateKey, "SHA-256", rng);
auto serverCert = rootCA.sign_request(req, rng, todayDate, expireDate);
/**
* Load Cert to In-Memory Database
**/
certificate cert;
cert.certs.push_back(serverCert);
cert.key = serverPrivateKeyShared;
creds.push_back(cert);
}
};
}; // namespace TLS
TLS::credentials globalCreds;
int main() {
globalCreds.createCert("www.google.com");
std::cout << "End" << "n";
return 0;
}

以下是Valgrind所指的植物库的函数，

void deallocate_memory(void* p, size_t elems, size_t elem_size)
{
if(p == nullptr)
return;
secure_scrub_memory(p, elems * elem_size);
#if defined(BOTAN_HAS_LOCKING_ALLOCATOR)
if(mlock_allocator::instance().deallocate(p, elems, elem_size))
return;
#endif
std::free(p);
}

Answer 1

植物学的作者回复我说

问题在于全局定义的对象。

问题是mlock池是在第一次使用时创建的单例池，然后在主返回后的某个时间销毁。首先创建对象。它分配内存。这将导致创建池。销毁发生后进先出法。因此，首先水池被摧毁。然后你的对象被破坏，并试图触摸已经被取消映射的内存(将其归零)。

解决方法，

• 创建Botan:：Allocator_Initializer对象以强制初始化在创建对象之前(因此池一直存在到对象已被破坏)
• 禁用locking_allocater模块
• 将环境变量BOTAN_MLOCK_OOL_SIZE设置为0
• 没有全局变量

原则上，锁定分配器不是对内存进行免疫映射，而是将其归零，并让操作系统在进程退出时取消映射。这可能仍然会破坏不变量，但不会那么糟糕。它还导致valgrind报告泄漏，这是令人讨厌的。

我认为因为它是直接被mmap处理的，而不是通过malloc，valgrind没有跟踪它。

Answer 2

全局变量，尤其是singleton，是多线程、复杂应用程序的祸害。这种设计总是会遇到这样的问题。

我通常会这样做：所有全局的东西都被定义为main或某个子函数中的局部变量，按适当的顺序，这样它就会按适当的相反顺序被销毁。类似依赖注入的技术可以用于在"依赖"的情况下传递那些对象；几乎所有的"；取决于他们。我很痛苦地意识到，这基本上是在大型复杂应用程序中可调试的唯一方法(想想应用程序本身和它在C++库之外使用的几十个库之间的2M loc)。在全局变量从定制代码中删除，然后从几个有问题的库中删除后；关闭时死亡"；几乎消失了。我不能保证它能解决每个人的问题——因为人们在提出新问题时可以很有创意——但这是IMHO朝着正确方向迈出的一步。

Answer 3

这是静态取消初始化顺序"fiasco"的一个示例。

有一些技术可以防止这种情况，但当你链接库时，它们可能不起作用，因为你无法控制它们的生存时间。

因此，最好的解决方案可能是在程序退出之前，在main的末尾或atexit函数中显式清除globalCreds的内容。其次，如果不需要清理，最好是泄漏结构。

如何从isocpp 中泄漏示例

TLS::credentials& x() {
static TLS::credentials* creds = new TLS::credentials();
return *creds;
}
TLS::credentials &globalCreds = x();

是的，这也冒犯了我的整洁感。

但这只是解决方法，globalCreds应该在main中创建，并传递给需要它作为引用的类(这些类也是在Creds之后的main中创建的)。