https://trunc.org/learning/cisco-hack-tracks-left-in-the-logs [trunc-logo-wide] Menu * About * Log analysis * Learning * Pricing * Login Try Free Here The Cisco Hack - Learning from the Cisco Hack, Trail of Evidence Found in Logs Cisco Talos recently shared a very interesting article detailing how Cisco was compromised. Companies rarely provide this level of detail, and when they do it serves as a great learning opportunity. These details are instrumental in helping the community get better by helping us understand TTPs being employed by bad actors, but also by highlighting the things we should be monitoring to help keep our enviornments safe. It also provides a great playbook other organizations can use to a) identify issues, b) how to mitigate and c) how to handle a security event. We thank the Cisco team for how they shared, and what they shared. Before reading more, we encourage you to read their report first: Cisco - Recent Cyber Attack 2022. As researchers, we were able to use the scenario they described to replicate the events in our labs in an effort to continue to the education and help the community as a whole. We focus on the traces of evidence left in the logs and how to make sense of what is going on. This should help security teams detect and mitigate similar issues if it effects their organization. Through the use of an effective logging strategy and platform you should be able to identify a series of events, tracks left by bad actors, and alert on key events that can be strong indicators of a compromise (IoC) allowing your team to respond quickly. 1- Password compromise + Phishing success The attackers were able to steal the Cisco VPN password from an employee through a compromise of that employee's personal Google account. Password sync was enabled in Google Chrome, giving the attacker access to all their passwords. They phished the employee using a series of phishing techniques to get their MFA token, the one that seemed to be most successful was voice phishing (vphish), but more on this in another post. Up to this point, what could Cisco have really done when the vector was out of their control? Let's assume, not much - short of Phishing awareness in their training. Let's also assume that Phishing is a tough thing to prevent (as it is) and is especially more complicated when we're talking about personal accounts and devices. It is why humans are, and will continue to be, the weakest link in the security chain. So up to this point, there is little that we could have done as defenders. Ok, an employee fell victim to a Phishing event. No way we could have detected that, they now have access to our network. What's next? How do you detect it? How do you minimize the damage? the impact? 2- New service (remote control tools) installed According to Cisco Talos, as soon as the attacker got in, they installed a number of tools for remote access: "The actor in question dropped a variety of tools, including remote access tools like LogMeIn and TeamViewer, offensive security tools such as Cobalt Strike, PowerSploit, Mimikatz, and Impacket, and added their own backdoor accounts and persistence mechanisms. " When you install an application, Windows will create a new log event (event ID 7045) saying that a new service was installed. Here are some more details on this event ID for Windows: Event ID Category Description Event ID 7045 System A new service was installed in the system. Here is an example of the event and what it might look like: 2022/08/11 18:48:44 WinEvtLog: System: INFORMATION(7045): Service Control Manager: truncroot: Windows-Trunc: Windows-Trunc: A service was installed in the system. Service Name: TeamViewer Service File Name: "C:\Program Files\TeamViewer\TeamViewer_Service.exe" Service Type: user mode service Service Start Type: auto start Service Account: LocalSystem If users are not allowed to install services on the internal servers, systems (which they probably should not), you can alert against these events. 3- New User Added Next, the attackers added new users to the system: "Consistent with activity we previously observed in other separate but similar attacks, the adversary created an administrative user called "z" on the system using the built-in Windows "net.exe" commands. This account was then added to the local Administrators group." This is another activity that generates logs that can be alerted in real time. Nobody should be adding new users without going through the proper channels, so that's an easy one to keep an eye out for and alert on if triggered. Windows will use event ID 4720 for new users. Event ID Category Description Event ID 4720 Account management A user account was created Here is an example of the event and what it might look like: 2022/08/11 19:07:51 WinEvtLog: Security: AUDIT_SUCCESS(4720): Microsoft-Windows-Security-Auditing: A user account was created. Account Name: z Account Domain: Windows-Trunc ... Windows will also log event ID 4732 for user group changes that can be used to detect whenever a user is added to an admin (or security-enabled) group. Event ID Category Description Event ID Account A member was added to a security-enabled 4732 management local group Here is an example of what a trigger on that event looks like inside of the Trunc logging platform: Trunc | Windows Event ID 4720 - New user added Note that Windows logs the subject (who did the change) and the target. In our tests, user truncroot did the change. That can also help filter events when only certain administrators can add/delete users. On AD networks, local user creations should be flagged for extra attention. This seems to be what they did, as it was the escalation of privileges that alert the securtiy team to an issue: "The attacker then escalated to administrative privileges, allowing them to login to multiple systems, which alerted our Cisco Security Incident Response Team (CSIRT).. " 4- Event log cleared After adding new users, modifying groups, and installing new software, the attackers tried to clear their tracks by modifying the event log. A great example of why remote logging is so important. "The attacker also took steps to remove evidence of activities performed on compromised systems by deleting the previously created local Administrator account. They also used the "wevtutil.exe" utility to identify and clear event logs generated on the system. wevtutil.exe el wevtutil.exe cl [LOGNAME]" But, if you don't have remote logging in place, Windows does create an event for this: event ID 1102 for security, so you can at least see that it was cleared. We can not emphasize how important it is track these and alert on them when triggered. Event ID Category Description Event ID 1102 Non Audit The audit log was cleared Here is what that log looks like: 2022/08/11 19:26:47 WinEvtLog: Security: AUDIT_SUCCESS(1102): Microsoft-Windows-Eventlog: no_user: no_domain: Windows-Trunc: S-1-5-21-3212774793-2808748914-3817205311-500 truncroot Windows-Trunc 0x2b22a64 Again, a great example of the importance of remote logging that can help centralize all your logs and maintain its integrity, preventing log tampering and missing information. 5- User deleted After clearing the logs, they deleted the users they had created: we observed the attacker removing the previously created local administrator account. net user z /delete This event is also logged by default in Windows with event ID 4726. If users should not be randomly deleted on your organization (which should not be), that's a good event the monitor: Event ID Category Description Event ID 4726 Account Management A user account was deleted Here is what that log looks like: 2022/08/12 19:24:26 WinEvtLog: Security: AUDIT_SUCCESS(4726): Microsoft-Windows-Security-Auditing: no_user: no_domain: Windows-Trunc: A user account was deleted. Account Name: z Account Domain: Windows-Trunc Note that Windows also logs who did the change. If only some users are allowed to delete accounts, the events from these users can be ignored to minimize the amount of log noise. 6- Firewall rule modified To enable their remote access, the attackers modified the Windows Firewall to allow their connections: "We observed them modifying the host-based firewall configurations to enable RDP access to systems. netsh advfirewall firewall set rule group=remote desktop new enable=Yes" Which is logged by default under Windows/Windows Firewall with Advanced Security event ID 2004. The only problem with this event ID is that it is very noisy. Event ID Category Description Event ID Security This event is logged when a rule has been added to 2004 the Windows Firewall exception list. Here is what that log looks like: Trunc | Windows Event ID 2004 - Firewall rule changed Even in our test system, it has generated over hundreds of logs in the past few hours. However, it is another thing to pay attention to, even if not the best rule to setup automatic alerts for. What The Cisco Hack Teaches us About Events At the core of the message is how important it is to have a good system in place to identify, notify and response to events. There team was able to effetively mitigate a security event because they had systems in place, but more importantly because someone was paying attention to the alerts. From this one event, and from their awesome disclosure, we can improve our security by tuning our environments to pay special attention to the following window events: 7045, 4720, 4732 and 1102. What other events can we connect to their disclosure? Would love more ideas to improve our list here. Thanks for that! Posted in security_research log-management by trunc_team Logging Guides We love logs. In this section we will share some articles from our team to help you get better at logging. Trunc Logging Logging for fun and a good night of sleep. * Real time search * Google simple * Cheap * Just works * PCI compliance Latest articles Latest articles from our learning center. 2022-08-12 The Cisco Hack - Tracking the Attack Through your Logs 2022-08-08 AWS Credentials Scan 2022-08-02 The Mozlila User Agent Bot 2022-07-24 Investigating a Hacked Linode server 2022-07-21 Log Management Challenges 2022-07-18 Linux read-only filesystem errors 2022-07-18 Critical Logs to watch: PHP Fatal errors 2022-07-05 HTTP Flood - DDoS analyzed 2022-06-30 HTTP User Agents 2022-06-28 Everything you don't need to know about NGINX error logs 2022-06-06 Web Interface for OSSEC 2022-06-05 Brute force attacks against Windows Remote Desktop 2022-06-02 Syslog Daemons difference (syslogd, rsyslog and syslog-ng) 2022-06-02 A Guide to Dropbear Logs 2022-06-02 Learn How to Test System Logging with Logger 2022-06-02 A Guide to Ubuntu Linux Logging 2022-06-02 A Guide to NGINX Logs 2022-06-02 Trunc - What Events to Log to your central logging server 2022-06-02 A Guide to Apache Logs 2022-06-02 Troubleshooting Remote Syslog with TCPDUMP Contact us! Do you have an idea for an article that is not here? See something wrong? Contact us at realpeople@noc.org Tired of price gouging Clear pricing. No need to guess. Real people. Real logging. [connection] Simple, affordable, log management and analysis. 14 days free trial. No credit card required. Get started TRUNC Trunc, a NOC.org company, focuses on making log analysis, storage and management simple and affordable for everyday network and security administrators. Articles Supported Logs PCI Compliance OSSEC Web Interface NGINX Log Analysis Log Management & Compliance Useful links Pricing Learning Logs Log Analysis What is this log? Help Documentation About Contact Us twitter: @logwithtrunc email: support@noc.org (c) 2022 Copyright: Trunc.org | Privacy | Terms & Conditions