Anyconnect Memory Locks up and Cert8

/in , /by

AnyConnect Locks up on Linux before it finally connects because of high memory usage that could go up to 100%. Machine would need to be rebooted to recover. Kubuntu, Ubuntu and Linux.

Scenario:

When using the Linux Cisco AnyConnect client x64 (like Kubuntu), memory usage gradually starts going up until it’s all used up. Anyconnect won’t connect.

I ran the VPN client within gdb to try and get a sense of what it was doing when trying to allocate so much memory.

Thread 3 of the process below is the only active thread and you can see that it is performing certificate related activities. Below are more related logs that I have collected

 

(gdb) thread 3
[Switching to thread 3 (Thread 0x7ffff28b3700 (LWP 3656))]
#0 0x00007ffff49f8180 in PR_Free () from /usr/lib/x86_64-linux-gnu/libnspr4.so
(gdb) bt
#0 0x00007ffff49f8180 in PR_Free () from /usr/lib/x86_64-linux-gnu/libnspr4.so
#1 0x00007ffff35bab1a in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#2 0x00007ffff35bcb15 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#3 0x00007ffff35be6eb in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#4 0x00007ffff35c3880 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#5 0x00007ffff35c413b in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#6 0x00007ffff35c41e7 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#7 0x00007ffff35bf262 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#8 0x00007ffff35bf751 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libnssdbm3.so
#9 0x00007ffff3a9c8e2 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so
#10 0x00007ffff3a87e75 in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so
#11 0x00007ffff3a8c78c in ?? () from
/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so
#12 0x00007ffff52b0d60 in ?? () from /usr/lib/x86_64-linux-gnu/libnss3.so
#13 0x00007ffff52aa4d2 in ?? () from /usr/lib/x86_64-linux-gnu/libnss3.so
#14 0x00007ffff5272091 in CERT_GetCertNicknames () from /usr/lib/x86_64-linux-gnu/libnss3.so
#15 0x00007ffff5272149 in CERT_FindUserCertsByUsage () from /usr/lib/x86_64-linux-gnu/libnss3.so
#16 0x00007ffff7844cc5 in CNSSCertStore::Enumerate(eCertType,
std::list<CCertificate*, std::allocator<CCertificate*> >&) () from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#17 0x00007ffff7818474 in CCollectiveCertStore::Enumerate(eCertType,
std::list<CCertificate*, std::allocator<CCertificate*> >&) () from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#18 0x00007ffff781363c in CCertStore::GetCertificates(CERT_ENTRY*,
CCertNameList*, std::list<CCertificate*, std::allocator<CCertificate*>
>&) ()
from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#19 0x00007ffff7818575 in
CCollectiveCertStore::GetCertificates(CERT_ENTRY*, CCertNameList*, std::list<CCertificate*, std::allocator<CCertificate*> >&) ()
from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#20 0x00007ffff780fb1c in
CCertHelper::GetClientCertificates(CERT_ENTRY*, CCertNameList*, std::list<CCertificate*, std::allocator<CCertificate*> >&, unsigned int) ()
from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#21 0x00007ffff7abb5d9 in ApiCert::getCertList(CERT_ENTRY*, CCertNameList*, std::string const&, ConnectProtocolType) () from /opt/cisco/anyconnect/lib/libvpnapi.so
#22 0x00007ffff7abbcb5 in ApiCert::getCertList(CERT_ENTRY*, std::string const&, ConnectProtocolType) () from /opt/cisco/anyconnect/lib/libvpnapi.so
#23 0x00007ffff7ace406 in ConnectMgr::resetCertRegistration(std::string) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#24 0x00007ffff7ae565b in ConnectMgr::setConnectionData(std::string
const&) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#25 0x00007ffff7aed002 in ConnectMgr::initiateConnect(std::string
const&, bool) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#26 0x00007ffff7af3c1a in ConnectMgr::run() () from /opt/cisco/anyconnect/lib/libvpnapi.so
#27 0x00007ffff7ac608a in ApiThread::threadProcedure(void*) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#28 0x00007ffff6753e9a in start_thread (arg=0x7ffff28b3700) at
pthread_create.c:308
#29 0x00007ffff5c6eccd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:112

SYSLOG logs:

std::list<CCertificate*, std::allocator<CCertificate*> >&) () from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#18 0x00007ffff781363c in CCertStore::GetCertificates(CERT_ENTRY*,
CCertNameList*, std::list<CCertificate*, std::allocator<CCertificate*>
>&) ()
from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#19 0x00007ffff7818575 in
CCollectiveCertStore::GetCertificates(CERT_ENTRY*, CCertNameList*, std::list<CCertificate*, std::allocator<CCertificate*> >&) ()
from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#20 0x00007ffff780fb1c in
CCertHelper::GetClientCertificates(CERT_ENTRY*, CCertNameList*, std::list<CCertificate*, std::allocator<CCertificate*> >&, unsigned int) ()
from /opt/cisco/anyconnect/lib/libvpncommoncrypt.so
#21 0x00007ffff7abb5d9 in ApiCert::getCertList(CERT_ENTRY*, CCertNameList*, std::string const&, ConnectProtocolType) () from /opt/cisco/anyconnect/lib/libvpnapi.so
#22 0x00007ffff7abbcb5 in ApiCert::getCertList(CERT_ENTRY*, std::string const&, ConnectProtocolType) () from /opt/cisco/anyconnect/lib/libvpnapi.so
#23 0x00007ffff7ace406 in ConnectMgr::resetCertRegistration(std::string) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#24 0x00007ffff7ae565b in ConnectMgr::setConnectionData(std::string
const&) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#25 0x00007ffff7aed002 in ConnectMgr::initiateConnect(std::string
const&, bool) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#26 0x00007ffff7af3c1a in ConnectMgr::run() () from /opt/cisco/anyconnect/lib/libvpnapi.so
#27 0x00007ffff7ac608a in ApiThread::threadProcedure(void*) ()
from /opt/cisco/anyconnect/lib/libvpnapi.so
#28 0x00007ffff6753e9a in start_thread (arg=0x7ffff28b3700) at
pthread_create.c:308
#29 0x00007ffff5c6eccd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:112

The relevant portion of the syslog follows. Note that the kernel experiences a page allocation failure shortly after the VPN connection is initiated.

Oct 31 15:08:01 Kubuntu acvpncli[1509]: Initializing vpnapi version
3.1.04072 ().
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: loadProfiles File:
../../vpn/Api/ProfileMgr.cpp Line: 100 No profile is available.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: getCurrentState
File: ../../vpn/Api/ClientIfcBase.cpp Line: 2058 API service not ready Oct 31 15:08:01 Kubuntu acvpncli[1509]: Current Preference Settings:
ServiceDisable: false CertificateStoreOverride: false CertificateStore:
All ShowPreConnectMessage: false AutoConnectOnStart: false
MinimizeOnConnect: true LocalLanAccess: true AutoReconnect: true
AutoUpdate: true ProxySettings: Native AllowLocalProxyConnections: true
PPPExclusion: Disable PPPExclusionServerIP: EnableScripting: false
TerminateScriptOnNextEvent: false AuthenticationTimeout: 12
IPProtocolSupport: IPv4,IPv6 AllowManualHostInput: true
BlockUntrustedServers: false PublicProxyServerAddress:
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
OnNegotiateMessageTypesComplete File: ../../vpn/Api/ApiIpc.cpp Line: 726 Master Agent Connection started.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: VPN state: Disconnected Network state: Network Accessible Network control state: Network Access:
Available Network type: Undefined
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
setConnectRequestComplete File: ../../vpn/Api/ConnectMgr.cpp Line: 9133 Connect request complete. Proceeding to cleanup.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
activateConnectEvent File: ../../vpn/Api/ConnectMgr.cpp Line: 1352 NULL object. Cannot establish a connection at this time.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Message type information sent to the user: Ready to connect.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: attach File:
../../vpn/Api/ClientIfcBase.cpp Line: 629 Client successfully attached.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: WMHintCB File:
../../vpn/Api/ClientIfc.cpp Line: 146 User did not implement WMHintCB.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: WMHintCB File:
../../vpn/Api/ClientIfc.cpp Line: 146 User did not implement WMHintCB.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: An SSL VPN connection to MyVPN.Server.com has been requested by the user.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
getProfileNameFromHost File: ../../vpn/Api/ProfileMgr.cpp Line: 793 No profile available for host MyVPN.Server.com.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: getHostInitSettings
File: ../../vpn/Api/ProfileMgr.cpp Line: 873 Profile () not found. Using default settings.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
deliverWebLaunchHostCB File: ../../vpn/Api/ClientIfc.cpp Line: 152 User did not implement deliverWebLaunchHostCB.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: loadProfiles File:
../../vpn/Api/ProfileMgr.cpp Line: 100 No profile is available.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
getProfileNameFromHost File: ../../vpn/Api/ProfileMgr.cpp Line: 793 No profile available for host MyVPN.Server.com.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Using default preferences.
Some settings (e.g. certificate matching) may not function as expected if a local profile is expected to be used. Verify that the selected host is in the server list section of the profile and that the profile is configured on the secure gateway.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function:
getProfileNameFromHost File: ../../vpn/Api/ProfileMgr.cpp Line: 793 No profile available for host MyVPN.Server.com.
Oct 31 15:08:01 Kubuntu acvpncli[1509]: Function: getHostInitSettings
File: ../../vpn/Api/ProfileMgr.cpp Line: 873 Profile () not found. Using default settings.
Oct 31 15:08:03 Kubuntu kernel: [395314.511185] kworker/u:2: page allocation failure: order:1, mode:0x4020

 

Resolution:

After troubleshooting, it turned out that Firefox cert8.db might have been corrupted (Anyconnect relies on some components of Firefox especially certs), In the case above, anyconnect is just not liking something about that file on this particular machine, or the issue might be an undocumented bug in Anyconnect on Kubuntu or Linux in general

Renaming the following two files allowed me to correct the issue (make sure you close Firefox first)

/home/Kubuntu/.mozilla/firefox/8n6w3k0u.default/cert8.db

/home/Kubuntu/.mozilla/firefox/8n6w3k0u.default/key3.db

After you rename the two files, start Firefox once and it will auto-recreate them.

Note: those files are profile specific, i.e. if you login to the machine with a different user and start Anyconnect, you might not face the same issue.

If you find this article helpful, please send us a note to Mike@bostonIT.com so I can keep on adding quality hands-on articles.