Koen Rouwhorst

personal blog

FLYING PIG: GCHQ's TLS/SSL knowledge base

Published on December 17, 2013

Documents from the ICTR-NE (Information and Communications Technology Research - Network Exploitation) organization at the GCHQ show that it operates a program under the name FLYING PIG that provides analysts with information about secure communications over TLS/SSL. The primary motivation for this program was the increasing use of TLS/SSL by GCHQ targets, according to one of the documents.

The documents, originally published by Brazilian TV program Fantástico in September, provide an insightful look into the program that allows analysts to query the vast repository of metadata about the world's secure communications. In this article, I describe the program on the basis of some actual screen captures of its interface.

FLYING PIG user interface

FLYING PIG user interface

In addition to "Query FLYING PIG", the user interface also shows two other tabs, "HRA Justification" and "Query QUICK ANT". HRA Justification, where the letters 'HRA' presumably stand for Human Rights Act 1998 (more details), is probably an interface that allows an analyst to provide a justification for FLYING PIG and QUICK ANT queries. The third tab, "Query QUICK ANT - Tor events QFD", where the letters 'QFD' stand for Question Focused Dataset, opens up an interface presumably similiar to that of FLYING PIG that allows an analyst to query QUICK ANT (Tor events). Neither HRA Justification and QUICK ANT are covered here. No further information is available on QUICK ANT.

FLYING PIG user interface

FLYING PIG user interface

Client

There are four ways to query FLYING PIG, by client, server, network or server certificate. In this section, I describe the client interface that allows one to query by client IP address (i.e., xx.xx.xx.127 in this example). It returns both detailed information about the client and the SSL servers it has visited.

Query by client IP

Query by client IP

General IP information

The first panel provides general information about the client, such as where it is geographically located and in which network it resides. The client IP address in this example resolves to "groupon.kr" and resides in the network of Korea Telecom. In the 'AS info' field you can find information about the IP address' associated autonomous system (AS), which is basically a network (or a group of networks) under a single administrative control. If the server had been a Tor node, the 'Tor' field would have shown details about this node from QUICK ANT.

General client information

General client information

Geolocation
Country: South Korea
City: Seoul
WHOIS
Network: xx.xx.xx.0/20
Network type: No results
Company: Korea Telecom
Domain: groupon.kr
Autonomous System information
Advertised by AS: 4766
Found within network: xx.xx.0.0/13
AS name: KIXS-AS-KR Korea Telecom
DNS
No results
Tor node
No matches

SSL servers visited

The othe client-specific panel shows all SSL servers the client has visited. In addition to server details, it shows the date of the first and last visit, the total number of visits and the pairing status (e.g. both directions, client-to-server, or server-to-client). The table is almost entirely comprised of visits to servers of internet company Mail.Ru. Also, note the visit to a Mozilla server (second-to-last row) where the server connected to this client (server-to-client), rather than the other way around (client-to-server). The IP address of the server resolves to "snippets.zlb.nl.mozilla.com" (Mozilla Snippet Service, think of about:home), which would explain the server-to-client pairing status.

Top SSL servers visisted

Top SSL servers visisted

Server IP Server country Server company info
94.100.184.14 Russia Mail.Ru
94.100.184.17 Russia Mail.Ru
94.100.184.16 Russia Mail.Ru
94.100.184.15 Russia Mail.Ru
[server IP] [server country] [server company info]
63.245.213.87 Netherlands Mozilla Corporation
94.100.181.127 Russia Mail.Ru
94.100.191.213 Russia Mail.Ru

Server

In this section, I describe the server interface that allows one to query by server IP (i.e., 94.100.184.14, a Mail.Ru server, in this example). It returns, amongst other information, details about the server, SSL traffic statistics, SSL certificates seen on this IP, and the top HTTP requests to this IP.

Query by server IP

Query by server IP

General IP information

This panel provides general information about the server, such as where it is geographically located (i.e., Moscow, Russia in this example), and in which network it resides. If the server happens to be a Tor node, the 'Tor' field shows further details about this node (populated from QUICK ANT). The server IP address in this example resolves to "mail.ru" and resides in the network of Mail.Ru, one of the world's largest internet companies. The fact that Mail.Ru is listed among internet companies like Facebook, Google, and Twitter in an NSA document about XKEYSCORE, underlines the significance of this company.

General server information

General server information

Geolocation
Country: Russia
City: Moscow
WHOIS
Network: 94.100.176.0/20
Network type: No results
Company: Mail.Ru
Domain: mail.ru
Autonomous System information
Advertised by AS: 47764
Found within network: 94.100.176.0/20
AS name: MAILRU-AS Limited liability company Mail.Ru
DNS
No results
Tor node
No matches

Top 10 SSL client geolocations

This bar chart shows the top 10 geolocations (in this case, countries, depicted by two-letter country codes) from which SSL clients connected to this server.

SSL client geolocations

SSL client geolocations

Top 10 SSL server ports

It may not come as a surprise that all clients connected to this SSL server through port 443, the default SSL port.

SSL server ports

SSL server ports

Top 10 SSL case notations

All intercepted signals get assigned a case notation that identifies the target being intercepted. The "overall" bar chart (left) shows the top 10 SSL case notations, each bar represents the relative number of signals intercepted for that case. I'm not sure what the "paired" bar chart (right) tells us.

SSL case notations

SSL case notations

SSL traffic statistics

This stacked area chart shows the SSL traffic statistics for the period of a week. It shows what percentage of the SSL traffic was comprised of client-to-server, server-to-client or bidirectional over time. In this example, the system has seen 104,317 unique SSL clients in one week (week ending December 23, 2011). Also, it mentions that 14.7% of the client-server IPs have seen traffic in both directions (both client-to-server and server-to-client).

SSL traffic stats

SSL traffic stats

SSL certificates seen on this IP

The table depicted below shows the SSL certificates seen on the specified IP address 94.100.184.14 (Mail.Ru). For some reason, once a VKontakte certificate was seen on this IP (VKontakte is partly owned by the Mail.Ru).

SSL certificates seen on this IP

SSL certificates seen on this IP

First seen Last seen Count Subject common name Issuer common name
2011-09-22 2011-11-25 > 2.3 million *.mail.ru Thawte SSL CA
2011-08-08 2011-11-05 > 1.4 million *.mail.ru Thawte SSL CA
2011-11-16 2011-11-16 1 *.vkontakte.ru Go Daddy Secure Certificate Authority

SSL pattern of life

Another panel shows the "average pattern of life" for a client, based on SSL events to the specified server. The table depicted below shows the top HTTP events to this server, ordered by percentage of occurrences.

SSL events

SSL events

SSL events

SSL events (continued)

Correlated event Event IP Event port Percentage
occurrences
of event
GET request to top3.mail.ru 217.69.135.12 80 29.1
GET request to top5.mail.ru 217.69.135.13 80 15.1
GET request to de.c2.bf.a1.top.mail.ru 217.69.134.253 80 14.2
GET request to my.mail.ru 94.100.184.40 80 13.2
GET request to my.mail.ru 94.100.184.41 80 12.9
GET request to stat.my.mail.ru 94.100.184.41 80 10.8
GET request to stat.my.mail.ru 94.100.184.41 80 10.5
GET request to mrimraker1.mail.ru 94.100.189.183 80 10.4

Top 100 SSL clients of a server

This server-specific panel shows the top 100 SSL clients that have connected to this Mail.Ru server (94.100.184.14). One can apply a filter to explicitly include or exclude certain countries by entering the countries' two-letter codes, separated by underscores. In this example the filter "GB_US_CA_NZ_AU" is applied to exclude SSL clients from both Great Britain, the United States, Canada, New Zealand and Australia. Another filter mentioned in the instructional text is "PK_IR_IQ" which can be applied to include or exclude clients from both Pakistan, Iran and Iraq.

Top SSL clients of a server

Top SSL clients of a server

Client IP Client country Client company
xx.xx.xx.212 Spain Telefónica de España, S.A.U. (rima-tde.net)
xx.xx.xx.1xx Spain R Cable Y Telecomunicaciones Galicia, S.A. (mundo-R)
xx.xx.xx.111 Germany Bertelsmann IT
xx.xx.xx.56 Norway Telenor Nextel AS (telenor.net)
xx.xx.xx.38 [client country] Vodafone ISP
xx.xx.xx.114 Germany Bertelsmann IT
[client IP] [client country] [client company]
[client IP] [client country] [client company]
xx.xx.xx.152 Ecuador EcuadorTelecom S.A.
xx.xx.xx.186 Ireland Vodafone ISP
xx.xx.xx.9 Malaysia TMNET
[client IP] South Korea [client company]
xx.xx.xx.53 Malaysia Core IP Development
[client IP] [client country] [client company]
xx.xx.xx.41 Ireland UTV plc
[client IP] [client country] [client company]
xx.xx.xx.38 Brasil Comitê Gestor da Internet no Brasil
xx.xx.xx.87 South Korea Korea Telecom
xx.xx.xx.156 South Korea Korea Telecom
xx.xx.xx.1 Ireland Vodafone ISP

Network

The network interface can be used to query by network (e.g. by CIDR) to show all SSL clients and servers present in the network or all HTTP requests that are made to IP addresses residing in the specified network (xx.xx.xx.0/24).

Query by network

Query by network

General network information

If available, this panel shows general information about a network, such as where it is geographically located. The network specified in this example is located in Seoul, South Korea.

General network information

General network information

Geolocation
Country: South Korea
City: Seoul
WHOIS
Network: No results
Network type: No results
Company: No results
Domain: No results
Autonomous System information
Advertised by AS: No results
Found within network: No results
AS name: No results

SSL clients in network

This panel shows all the SSL clients that were, at some point, present in the specified network. Note that the header of the second column mentions that the client company information comes from "GEOFUSION", which we know from a CSEC presentation (8th item in the list) is a repository that contains geolocation information.

SSL clients in network

SSL clients in network

Client IP Client company information First seen Last seen
[client IP] Korea Telecom: mailplug.co.kr 2011-09-04 2011-09-04
[client IP] [client company information] 2011-10-26 2011-11-23
[client IP] [client company information] 2011-10-22 2011-10-22

Certificate

The server certificate interface allows one to query by certificate metadata, e.g. by 'subject', 'issuer' or 'RSA modulus' (component of a public key). The default results view shows both the matching HTTP requests and server certificates.

Query by certificate metadata

Query by certificate metadata

HTTP requests

This panel shows all HTTP requests that match the query "%mail.ru". The columns 'First seen' and 'Last seen' respectively show the first and last date an HTTP request was seen to a host. The other two columns, 'Count w/o 25th new' and 'Count all time', show the total number of HTTP requests to a host for the last week (e.g. week of 25th of November) or for all time respectively. For example, in a six-week period in end of 2011, more than 42 million requests were seen to only one host (94.100.184.105, swa.mail.ru).

Matching HTTP requests

Matching HTTP requests

Server IP Host name First seen Last seen Count
94.100.184.105 swa.mail.ru 2011-10-13 2011-11-25 > 42 million
94.100.184.104 swa.mail.ru 2011-10-13 2011-11-25 > 36 million
217.69.135.201 fc.ef.d4.cf.bd.a1.top.mail.ru 2011-10-13 2011-11-25 > 16 million
217.69.135.13 top5.mail.ru 2011-10-14 2011-11-25 > 14 million
217.69.135.12 top3.mail.ru 2011-10-14 2011-11-25 > 12 million

Certificates

All certificates that match the query are depicted in the certificates panel. The 'basic' view (which is the default) shows for each certificate the validity period, the number of times it was served, subject and issuer details, and whether it was self-signed. Right-clicking a row opens up another panel on the right that lists all server IPs that serve (or have served) the selected certificate. There's also an 'advanced' view which adds RSA modulus (component of a public key) and cipher suite distribution details.

Certificates

Certificates

Valid from Valid to Count Subject Issuer Self signed
2011-01-31 2012-03-27 > 15 million *.mail.ru
Russia
LLC Mail.Ru
Thawte SSL CA
United States
Thawte, Inc.
No
2010-01-21 2011-01-20 > 1 million *.mail.ru
Russia
LLC Mail.Ru
Thawte Premium Server CA
South Africa
Thawte Consulting cc
No
2011-09-25 2013-11-25 > 30,000 *.money.mail.ru
Russia
LLC Mail.Ru
Thawte SSL CA
United States
Thawte Inc.
No
2010-01-25 2012-01-27 > 8,000 mail.ru.is
Iceland
mail.ru.is
[common name]
United States
[organization name]
No
2011-03-04 2012-03-03 > 1,000 [common name]
United States
[organization name]
[common name]
United States
[organization name]
Yes
2011-09-27 2012-09-25 > 1,000 mail.ru-com-ru
-
mail.ru-com.ru
Thawte DV SSL CA
United States
Thawte, Inc.
No
2010-02-12 2012-11-08 > 1,000 mx1.shogo-mail.ru
Russia
Shog
shogo.ru
Russia
Shogo
No
2011-09-15 2012-09-14 > 600 imgs.mail.ru
Russia
[organization name]
[common name]
[country]
[organization name]
No
2011-10-05 2014-10-04 > 300 [common name]
Russia
[organization name]
mail.ru
[country]
mail.ru
Yes
2011-09-15 2012-09-14 > 200 auth.mail.ru
Russia
[organization name]
[common name]
[country]
[organization name]
No

Conclusion

FLYING PIG is a program that allows analysts to query GCHQ's vast repository of metadata about the world's secure communications over TLS/SSL. It's certainly not a program through which the GCHQ, or NSA for that matter, performs man-in-the-middle attacks against internet services like Google, as reported by others, including Bruce Schneier. The reports that claim the NSA performed MITM attacks against Google are based on a small piece of a document that describes a FLYING PIG (which is a not an NSA program, as you may have noticed) use case (presumably, an investigation into the DigiNotar CA breach). That's not to say the GCHQ doesn't perform MITM attacks, but there's no evidence to be found in this document. Though, FLYING PIG may be used to prepare MITM attacks, e.g. by providing information about a target.