Watch-out for compromised third party accounts and bad Active Directory configuration.
This week during the SSTIC2017 annual cyber security conference, a French conference running consecutively since 2004, the National Cybersecurity Agency of France (ANSSI) gave a presentation detailing their 2015 audit of their investigation and remediation of the intrusion which affected TV5Monde television network channel. This intrusion was allegedly conducted by the Fancy Bear/APT28 actor, and resulted into broadcasting and social media sabotage.
Although, this happened two years ago — hats off to both ANSSI and TV5Monde for sharing their experience, what they have learned and their methodology during the investigation. Very few companies understand the importance of sharing such information in order to prevent similar scenarios. This sort of feedback is incredibly valuable and informative for the community. Thanks.
You can find the original video onlinebut since it is in French, I decided to make a quick transcription of the main lessons and points from the presentation including some personal notes on the incident.
Defining the next 48 hours post incident
- Get in touch to define the goals and the point of contacts for the different actors — happened within hours of the attacks.
- Look for “quick wins” to bootstrap the investigation.
- Look for malwares or rootkits that would damage the broadcasting.
- Define the team (Technical Coordinator, 5 Forensics Analysts, 1 Reverse Engineer, 2 Network Analysts)
Quick-Wins
The first artifact detected was the presence of an Administrator account with an English username — which was very surprising for the auditors given the fact the whole Active Directory was in French.
LocalAdministrator
This account allowed to taint a machine, and to retrieve initial timestamp information.
This also allowed to identify a suspicious DLL (ConnectBack.DLL is an arbitrary name) on the active malicious session ran by rundll32.exe and C&C IP. This malicious DLL can then be analyzed to understand in depth what the malware is doing but also identify code similarities with other malwares.
Timeline of Response
9 April 2015 — Beginning of the incident response and remediation.
- Data Collection over 1–2 weeks.
- Remediation between 1–2 months.
- Analysis cycles over multiple months
- Reporting can be over multiple months.
Data Collection & Analysis
ANSSI describes they collected ~300GB of compressed logs for network logs (TACACS), Internal wiki logs (Apache logs), Firewall logs (ASA), Windows logs (Active Directory, Desktops & Servers) — in addition of ~13TB copy images of harddisk, memory (RAM) and embedded devices of the main target of interests.
ANSSI rightly focuses on the importance of the logs collection but also on memory forensics part which is very important in such scenarios to keep a frozen state of the infected or machines of interested but easily allows to retrieve information such as the quick-wins described above.
This is why at Comae we decided to build and we are currently working (AND still looking for beta testers to improve it!) a comprehensive and scalable platform such as Stardust for memory forensics for incident response & compromise assessment.
Goals
Multiple parties (TV5Monde, French Ministry of Interior, ANSSI, ENISA, and other television networks) were involved. Each involved party had different goals and expectations.
TV5Monde
- Scope Analysis
- Production recovery (Being able to broadcast again)
- Remediation
French Ministry of Interior
- What happened ?
- Timeline
ANSSI
- When ?
- How ?
- What ?
Partners (ENISA) & Television Networks
- Awareness and Modus Operandi
- Indicator of Compromise (IOCs)
- Prevention
Timeline Of Attack
Initial Access & Compromission
The attacker got his initial access the network on the 23rd January 2015 and explored it over multiple weeks.
One of TV5Monde multimedia server (used by journalists to send content back) had its RDP port exposed to internet and was using default username/password. But this machine was not connected to the internal network, and was quickly classified as dead-end by the attacker.
The attacker came back later on, this time, with a compromised third-party account to connect through the TV5Monde VPN before compromising it on the 6th February 2015 over a one week period, and discovered two machines (ROB1 & ROB2), after scanning its internal network, that were Windows machines managing the cameras.
Creation of LocalAdministrator account.
The attacker used one of these compromised machines (ROB2) to create a new Active Directory Administrator user (LocalAdministator) (11th February)
Collection & Verification
During the 16th February to 25th March 2015 period, the attacker searched (“telnet”, “ssh”, “video”, “compte”, “pass”, “VPN”, etc.) & collected data on the various internal platform such as the IT Internal Wiki and retrieved as much login and password information as possible and also spend the time to verify those information to make sure they were not expired or outdated.
Successful access to the Wiki and data extraction
The attacker compromised another administrator machine (Codenamed: ANKOU) which contains the Remote Access Control (RAT) which was used for the sabotage. Prior to this, the attacker also dropped njRAT as a decoy on the system but didn’t run it — ANSSI isn’t sure why.
Social media accounts got compromised few hours before the sabotage of the broadcasting network.
As we can see from the above information, the attacker was in the network for almost 3 months and carefully prepared his sabotage operation by verifying the collected information.
D-Day
Sabotage
At 19:57, the attacker did his first damaging operation by faulty re-configuring all the IP configuration of the media encoded. This misconfiguration only gets enabled when the technical teams reboot the machines.
At 20:58, the online presence is affected through social media accounts (YouTube, Facebook, Twitter) and the website of TV5Monde which is modified.
At 21:48, the attacker runs a series of destructive commands (extracted from TACACS logs) to erase the firmwares from the switches and routers that results into the black screens — except for one new channel that was launched on the same day which was covering the attack from inside.
Black Screens Of Death
10 Prevention & Remediation Measures
1. Centralize and capture all the network, servers and desktop logs.
Having centralized logs make the incident response step easier — and results in better quality analysis.
ANSSI also noted that TV5Monde very understanding of the importance of logs, which is not always the case of compromised companies. This is an incident response, therefore it is very important to be able to analyze all the logs to not miss anything but also be as quick and efficient as possible.
In the case of TV5Monde, ANSSI emphasized they had access to good quality logs.
2. Keep a certain level of control of the IT relevant to your organization, especially if you use third parties.
Outsourcing means it is very difficult to have enough the information required to take decisions — including re-configuring, collecting logs, isolating and take urgent decisions by yourself.
Why is it important ? Because this add a considerable delay between the time to decision and action — which is critical in such scenarios.
The Active Directory was composed of around:
- ~140 Servers —Windows & Linux mainly virtualized over ESXi
- 380 Windows desktops
- 310 Mac OS X
3. Build a filtered administrator network, issue dedicated administrator desktops and isolate service admin interfaces.
Red are privileged groups (Admins) — and blue is the actual admin rogue account.
This is after, the AD had been migrated to a more comprehensive and isolated version.
As you can see from the above screenshots limited the admin access, removing the unused support accounts and building a comprehensive AD is critical — but also documenting it to be able to keep track of its modification for the future too.
Sean Metcalf wrote a blogpost describing how to scan your Active Directory to detect priviledged accounts efficiently using PowerView from Will Schroeder.
4. Make sense of the issuance and filtering of privileges.
Each level is isolated from each other (unless a 0day happens :))
As part of the remediation, ANSSI also did a great work of Active Directory hardening focusing as you can see on the above screenshots on:
- Rationalizing the domain architecture logic.
- Deleting the unnecessary (or forgotten) accounts and groups. Active Directory is too easy to administrate which often results in the creation of overly privileged accounts. Active Directory Administration and Active Directory Security are two different specialties. Active Directory Security is too often an vacancy because it is considered too expensive and wrongfully unnecessary.
- Privileges per group.
- Authentication restrictions — to prevent self compromise.
- Password policy — many passwords haven’t been changed over many years.
Active Directory Administration versus Active Directory Security
5. Privileged accounts should have limited accessibility but also usability. Using privilege accounts to browse the internet and office suite operation must be forbidden.
This makes sense but who knows, you want to avoid the account and machine administrating your hypervisor and domain controller to browse unnecessary websites to not increase your attack surface.
6. Prevent sensitive administration information to leak from a technical point of view by leveraging blacklisting connections.
- Dedicated and isolated administrator network.
- Filtering & Firewalls.
- Dedicated machines for third parties.
- RDP hardening with Shadowing.
7. Keep an up-to-date inventory of the account services and their applications.
Unfortunately, too often many CIOs don’t know exactly what applications are actually being used by their users — having worked on an application deployment solution (acquired and rebranded as VMware AppVolumes) we often bumped into that problem.
8. Keep an up-to-date and complete documentation of your IT infrastructure, its network and the different interaction.
You don’t want your attacker to end up with a better documentation than your CIO.
This will also save a lot of time to both the investigators and your organization when it comes to understanding the potential attack vectors, what happened but also taking decision (cf. #1). Unfortunately, most of companies have difficulties understanding this — and often have a flawed view of what their internal IT really looks like.
9. Regular security audits including compromise assessments.
Not a secret, you should test your own applications and network before an uninvited guest does it for you. According to Microsoft Advanced Threat Analysis Team, 146 is the median number of days an attacker resides within a network before detection.
10. Surround yourself of cybersecurity experts along your projects and be prepared to respond — but also let them work in peace.
If you already have trusted partners who know you — this will obviously make you better prepared and you won’t have to wait for quotes for days.
But something which was surprising was the fact the journalists themselves were so focused on the story they often prevented the incident responders to do their jobs — the speaker even mentioned they are to run away from the journalists, they got interrupted many times and got followed by cameras which was against their own interest.
Thanks to @SwitHak for bringing my attention on this presentation.
Congratulations again to the ANSSI Team for conducting the analysis and assisting in the Active Directory migration/remediation. Thanks again to TV5Monde & ANSSI for sharing those information with the public.
I personally think this shows great technical leadership from both of them, and I hope this will encourage more parties to mature their cyber security practices and do the same.
Information sharing is critical. This allow companies but also security experts to learn & understand to better analyze and prevent incidents.