Tor as infrastructure: challenges and opportunities

Infrastructure as a Service (IaaS) is commonly used within enterprise environments whereby organizations pay for someone else’s physical or virtual resources, including networking resources. Individuals also pay for IaaS when, for example, they need to deploy a VPS or VPN for personal use. From Gartner:

Infrastructure as a service (IaaS) is a standardized, highly automated offering, where compute resources, complemented by storage and networking capabilities are owned and hosted by a service provider and offered to customers on-demand.

 

Tor, the network of operators that make up the network, is IaaS. What is unusual about this terminology is that the Tor network is distributed, and utilization of this networking resource is free. Companies like Duck Duck Go, Facebook, New York Times, and every organization utilizing SecureDrop including the Associated Press, is using this IaaS in order to best support their users’ privacy rights.

A problem that Tor network operators have always faced is that setting up and maintaining the network is not free. Tor is free-to-use IaaS on purpose — people and services need to be able to use the network without attribution in order for Tor to provide specific guarantees of privacy.

If privacy infrastructure operators had better funding, we would be in a better position to support larger infrastructure needs. For example, if Mozilla or Brave ever wanted to collect browser telemetry over Tor onion services to best support the privacy of their users, the Tor network would likely need to pivot towards larger and more stable network operators. In this article, we will look at some of the challenges and opportunities for operators of privacy infrastructure.

Challenges

Emerald Onion was created, in part, to be able to demonstrate a successful Transit Internet Service Provider that only supports privacy IaaS. It was designed to best leverage existing laws in the United States in tandem with operational designs that require privacy-focused security properties. There have been and continue to be serious challenges facing Emerald Onion along with any other organization that is dedicated to privacy IaaS.

IP transit service is exclusively a for-profit business

IP transit is a required part of an ISP. Emerald Onion, Riseup, Calyx, Quintex, etc, need to pay upstream providers who provide the physical transport of the encrypted packets that we transit as part of the Tor network. This transit service is expensive. For example, 1Gbps of service in a residential setting can cost around $80 per month in Seattle, and 1Gbps of service in a datacenter can easily cost $800 per month. This dramatic cost difference is because of capitalism — it is presumed that a service provider in a datacenter environment is going to be profiting off of this service. Upstream providers don’t care one bit that Emerald Onion is a 501(c)3 not-for-profit supporting human rights.

Little options for trustworthy, open source hardware, particularly networking equipment

Emerald Onion is using general-purpose computing devices (currently low-power Intel Xeon D) with BSD operating systems. It is a priority for us to be using trustworthy compute infrastructure, so we are at least ensuring that the kernels and applications that we use are free/libre open source software. We hope to transition to free/libre open source hardware and firmware as soon as we can, but we also have to be concerned with compatibility and stability with HardenedBSD/FreeBSD, and the cost of this hardware. We know that options exist for free/libre open source hardware, but this is still a very new and maturing market. To further complicate this prioritized need for trustworthy compute infrastructure, Emerald Onion has particular interest in 10Gbps networking for both the LAN and WAN.

One day, we’d like to be able to support 40Gbps and 100Gbps; however, we are not aware of any free/libre open source hardware and firmware that supports 40Gbps or 100Gbps networking.

High cost of network redundancy

Our proof-of-concept work has focused on low-cost options. This means we do not currently have redundancy at our LAN or WAN layers. Network redundancy for Emerald Onion, at minimum, would entail having not just one expensive IP transit link, but two, ideally from different upstream providers, which means two edge routers and two links to each of our Internet Exchange Points. This would also mean that we have to add redundant LAN switching, and all of this means increasing our rack space and power requirements. Basically, we would have to more than double our recurring costs to be able to have this level of infrastructure stability. While Tor itself is highly resistant to network changes, the more capacity that Emerald Onion, and other large Tor operators support, the greater the negative impact we would impose on the Tor network whenever we have to perform hardware, firmware, kernel, and application updates.

IPv4 scopes for exit operators

As an exit relay operator, Emerald Onion must own and operate our own IPv4 address space for efficiently handling abuse communications from other service providers and law enforcement. Additionally, relay operators who peer directly with other service providers in Internet Exchange Points (IXPs) who have their own Autonomous System Number (ASNs) also require their own IP space. The entire world ran out of IPv4 address scopes to hand out to new and existing service providers a few years ago, and this is a blocker for any new Tor operator who is working to achieve the same level of stability that Emerald Onion is working to achieve.

Tor exit relaying currently depends in IPv4 connectivity between Tor routers (middle relay to exit relay traffic, for example). To be given an exit flag, a static IPv4 address is ideal for the Tor network (dynamic IP addresses would require a few hours delay of client discovery in the consensus).

Tor exit operators would not need their own Regional Internet Registry (RIR) -provisioned IPv4 address space if the exit flag could be given to IPv6-only operators, but this is not currently possible. ORPort connections (inter-tor circuit connections from middle relays, for example) cannot usually generate abuse, so IPv4 scope leasing is an easier option if IPv6-only exiting was a possibility.

One idea that Emerald Onion has had is that it may be possible to make proposals to large organizations, including universities, that are sitting on very large IPv4 scopes. We think that these organizations might be willing to donate small (/24) scopes to not-for-profit Tor network operators.

Opportunities

Surveillance and latency minimization

Seattle is home to a very large telecommunications hub called the Westin Building Exchange (WBE). We know that this building has National Security Agency (NSA) taps on I/O connections that are likely to facilitate traffic to regions like China and Russia. Additionally, WBE hosts several of the Internet’s DNS Root Servers, several of which are part of the Seattle Internet Exchange (SIX).

Emerald Onion went through the process of securing our own ASN, IPv6, and IPv4 scopes from American Registry of Internet Numbers (ARIN). We needed these things to connect to the SIX. Connecting to the SIX means that we are physically and directly connected to as many as 280 other service providers. We made this a priority because direct peering, using Border Gateway Protocol (BGP), minimizes the amount of clear-net switches and routers that a Tor user’s exit traffic has to travel through to reach its final destination. Every switch or router that Tor traffic has to traverse is an opportunity for surveillance and adds latency.

This strategy for Tor exit router placement is also ideal considering DNS. Being that multiple DNS Root Servers are directly peered with Emerald Onion, this further minimizes a global persistent adversary’s ability to spy on what Tor users are doing.

Statistically, due to requirements in the Tor protocol, individual Tor circuits bounce around multiple countries before they exit the network. This means that a non-trivial amount of the traffic that Emerald Onion, and any other United States exit operator facilitates, comes from a middle relay not within the United States. In tandem, generally speaking, a non-trivial amount of Tor exit traffic is destined to American services like Akamai, Cloudflare, Facebook, Google, and DNS Root Servers. These two likelihoods, together, means the following:

Tor exit traffic that is destined to service providers in the United States is best served, in terms of surveillance and latency minimization, by Tor exit operators that have exit relays connected to IXPs in datacenters along the coasts of the United States where undersea cables physically terminate, presuming that popular service providers like Akamai, Cloudflare, Facebook, Google, and DNS Root Servers are direct peers. This, in theory, minimizes the opportunity for American-sponsored traffic analysis, data retention, and surveillance, in addition to any other global persistent adversary who may have compromised network equipment within IXPs.

Emerald Onion has already compiled an initial list of IXPs around the United States. We continue to work on a list of qualities that an IXP should have that is a ideal for a Tor network operator:

  • Number of participants — This is important because if there is a peering link (using BGP) with another service provider, the opportunities for surveillance are minimized. For obvious reasons, the amount of direct peers is as important as the popularity of said services.
  • Access to specific participants — This is important because, for example, peering agreements with large providers such as Akamai, Cloudflare, Facebook, Google, and DNS Root Servers minimize the opportunity for surveillance while minimizing latency.
  • Nonprofit and affordable — A large number of IXPs are for-profit and thus have high up-front and high recurring costs for connectivity, in addition to setup fees and recurring fees for copper or fiber maintenance.
  • Geo-location — This is important because of, at least, location diversity, peer diversity, and direct connections with international undersea cables are focal points for the facilitators of global passive surveillance.
  • Prohibits network surveillance — The Seattle Internet Exchange, for example, has a stated policy that prohibits surveillance on peering links. One day, we hope that the SIX, and other public-benefit IXPs, will also publish a regular transparency report.

Funding

Emerald Onion has been in operation for 10 months. We wouldn’t exist, as we are today, without the generous startup grant of $5,000 from Tor Servers. We also would not still be around without the continuous donations from our Directors who personally donate as much as $350 each month. We currently require roughly $700 per month to operate, largely due to our service contract with our co-location provider who is also our upstream transit ISP.

Going back to the beginning of this article, the Tor network is a privacy-focused IaaS. Sustainability is a constant issue for Tor network operators, especially for operators who preemptively tackle legal and long-term operational challenges. We need help. There is no easy answer for funding. Grant writing and grant management is not a trivial task, nor is sustaining a 501(c)3 not-for-profit purely based on part-time volunteer work. Emerald Onion is incredibly lucky to have a few people who regularly donate large amounts of money and time to keep the organization online, but this is not sustainable.

The operations model that Emerald Onion has created, however, is scalable if properly funded. If we were provided between between $7,000 to $10,000 per month, we could multiply our capacity by a factor of 10. If we had a pool of funding that supported 10 independent Tor network operators in the United States (there are over 100 IXPs in the United States), we could dramatically bolster the capacity and stability of the Tor network while also minimizing network surveillance opportunities and network latency.

Conclusion

I hope that this article begins to shed light on the challenges facing privacy IaaS providers like the thousands of operators that make up the Tor network. Emerald Onion is going to continue to educate others on these topics, attempt to find and create solutions for these challenges, and continue to encourage hacker communities around the United States to build their own privacy-focused not-for-profit ISP.

Introducing gibson

Artwork by Mike Finch (CC BY 4.0)

 

As you may know, Emerald Onion systems run HardenedBSD. BSD systems in general, and HBSD in particular, provide numerous advantages to our team in operating secure and highly performant Tor relays. But BSD systems make up only a very small percentage of the Tor network. There are many similarities between BSD and Linux, with which many users may be more familiar, but the differences can be intimidating. We’re addressing this by launching gibson, a project to develop a suite of tools to address the needs of Tor service operators. The Tor network is more robust when it is diverse, and this is one way that we can encourage a more diverse Tor network and enhance our community.

It is important to say that while our initial focus is on BSD systems, our plan is to extend gibson to serve the Tor community regardless of platform. We’re starting with HBSD because it’s an obvious and natural choice for us; we believe in “dogfood“, and we want you to be assured that the code we share is used by our team in real deployments. In our mind it isn’t enough to make running Tor services easy, our tools must also help make services secure and reliable. At Emerald Onion, we do this by example.

What is gibson?

A generous description is that gibson is a no-dependency suite of cross-functional tools for creating and maintaining secure and robust Tor services. We currently support HardenedBSD systems, but plan to extend our support to FreeBSD, OpenBSD, and Linux in future releases.

We say that this is a generous description because we want our tools to take as light a touch as possible. To an experienced user, gibson may not appear to do much of anything at all. This is an intentional design decision. An experienced user might say, “Using gibson to do an update achieves the same outcome as just running these three commands I already know”. Truly, nothing would make us happier than this outcome. We want for everyone in the Tor community to be an expert of the tools that make the network operate, from Tor itself to the operating system, hardware, and everything in between. That said, we hope that experienced users will continue to use gibson, and that they will propose new solutions and new functionality.

The goal of gibson isn’t to make difficult Tor management tasks trivial. Rather, the goal of gibson is to make Tor management tasks consistent. We believe that secure systems are built around reproducible, auditable processes. Most of our maintenance is simple and mundane, but a small configuration error or a missed security patch could be catastrophic to the anonymity and security of Tor users. We want to eliminate possible sources of human error and to share our best practices with the community.

We also believe that users should be able to adopt gibson on existing systems without onerous effort, and should also be able to walk away from it whenever they wish without any lock-in. We want our tools to promote the adoption of correct processes and best practices. We also hope that they will be educational. Users new to BSD systems, new to Linux, and new to Tor should be able to look at our code and with minimal effort be able to understand what it does and why.

Finally, we say that gibson is cross-functional because our solution space is defined by the needs of a secure and robust Tor deployment. We do not seek to replace virtualization and jail tools (for instance, bhyve, virtualbox, ezjail, iocell, etc.). We do not seek to replace disk encryption tools (geli, LUKS, etc.). We’re not replacing any web servers, either (nginx, Apache, Caddy, etc.). What we are doing is providing tools to streamline the implementation of these other projects into a complete solution which addresses the needs of Tor administrators.

What does gibson do now?

Currently, gibson updates and controls Tor services running in HBSD jails. A simple and mundane task, but one that we want to make sure is done consistently during each of our maintenance windows. Our initial release of gibson is version 0.1, which is derived from a handful of scripts currently used in maintenance of Emerald Onion systems.

  • 0.0.1 Initial scripts used for EmeraldOnion system maintenance
  • 0.1.0 First version 0.0.1 scripts refactored into gibson: apply system and package updates in jails; start, stop, restart tor services in jails;

Where can I get gibson?

We’re working on getting gibson into the HardenedBSD ports repository, and it should land there shortly.  It will take a little more time for binary packages to become available for users who prefer to use pkg.

In the meantime, gibson is available at our GitHub, as is the files you need to sideload it as a port.  Detailed installation instructions are available there.

Roadmap (or: what will gibson do in the future?)

0.2 -> 0.5

  • Create and maintain template jail(s); clone templates into new jails and deploy tor services:
    • middle relays
    • exit relays
    • bridges
    • onion services (with nginix, initially)
  • Create and manage geli-encrypted ZFS pools
  • Initial creation of encrypted providers and pool members from specified devices
  • Replacement of devices due to failure or capacity expansion
  • Good ideas suggested (or implemented and submitted) by our community!

0.6 -> 0.9

  • Support for FreeBSD systems
  • Good ideas suggested (or implemented and submitted) by our community!

1.5+

  • Support for Linux systems
  • Support for non-geli encrypted filesystems
  • Support for non-ZFS storage pools
  • Good ideas suggested (or implemented and submitted) by our community!

House Style

gibson is always written in all-lowercase.

Logo

The gibson logo was generously donated by Mike Finch and is licensed by Emerald Onion as Creative Commons Attribution 4.0 International.

Emerald Onion’s Articles of Incorporation

The following is the text of the Articles of Incorporation for Emerald Onion that were filed with the Secretary of State for Washington State. We filed this with the state and verified it was received and accepted before filing IRS Form 1023-EZ.

Here is a link to the word document used to create it.

Nonprofit Corporation Articles of Incorporation of Emerald Onion

Pursuant to RCW 23B.02.020 of the Washington Business Corporation Act, the undersigned does hereby submit these Articles of Incorporation for the purpose of forming a nonprofit corporation.

Article 1: Name

The name of the corporation is Emerald Onion.

Article 2: Effective Date

The effective date of incorporation shall be 6/15/2017.

Article 3: Existence

The corporation shall have perpetual existence.

Article 4: Purpose

The purpose of the corporation is to promote and support online anonymity and privacy. Towards this purpose, the corporation shall engage in charitable, educational, and scientific purposes, including the making of distributions to organizations that qualify as exempt organizations under section 501(c)(3) of the Internal Revenue Code, or the corresponding section of any future federal tax code.

No part of the net earnings of the corporation shall inure to the benefit of, or be distributable to its members, trustees, officers, or other private persons, except that the corporation shall be authorized and empowered to pay reasonable compensation for services rendered and to make payments and distributions in furtherance of the purposes described in section 501(c)(3). No substantial part of the activities of the corporation shall be the carrying on of propaganda, or otherwise attempting to influence legislation, and the corporation shall not participate in, or intervene in (including the publishing or distribution of statements) any political campaign on behalf of or in opposition to any candidate for public office. Notwithstanding any other provision of these articles, the corporation shall not carry on any other activities not permitted to be carried on (a) by a corporation exempt from federal income tax under section 501(c)(3) of the Internal Revenue Code, or the corresponding section of any future federal tax code, or (b) by a corporation, contributions to which are deductible under section 170(c)(2) of the Internal Revenue Code, or the corresponding section of any future federal tax code.

Article 5: Distributions Upon Dissolution

Upon termination or dissolution of the corporation, any assets lawfully available for distribution shall be distributed to one or more qualifying organizations described in Section 501(c)(3) of the Internal Revenue Code, or the corresponding section of any future federal tax code, which have a charitable purpose which, at least generally, includes a purpose similar to the terminating or dissolving corporation.

Article 6: Initial Board of Directors

The corporation’s initial directors are as follows:

Christopher Sheats
815 1st Ave # 331
Seattle, WA 98104-1404

Christian Severt
815 1st Ave # 331
Seattle, WA 98104-1404

William Scott
815 1st Ave # 331
Seattle, WA 98104-1404

Article 7: Registered Agent and Office

The street address of the Registered Agent of the corporation is:

Registered Agent Solutions Inc
3400 Capitol Blvd SE Ste 101
Tumwater, WA 98501-3351

Registered agent mailing address:

Registered Agent Solutions Inc
PO Box 1368
Olympia, WA 98507-1368

Article 8: Principal Office

The corporation has a principal office. The street address of the principal office is:

815 1st Ave # 331
Seattle, WA 98104-1404

Article 9: Mailing Address

815 1st Ave # 331
Seattle, WA 98104-1404

Article 10: Indemnification

The corporation does indemnify any directors, officers, employees, incorporators, and members of the corporation from any liability regarding the corporation and the affairs of the corporation, unless the person fraudulently and intentionally violated the law and/or maliciously conducted acts to damage and/or defraud the corporation, or as otherwise provided under applicable statute.

Article 11: Incorporators

The names and addresses of the Incorporators are:

Christopher Sheats
815 1st Ave # 331
Seattle, WA 98104-1404

Christian Severt
815 1st Ave # 331
Seattle, WA 98104-1404

William Scott
815 1st Ave # 331
Seattle, WA 98104-1404

Matthew McCoy, J.D.
815 1st Ave # 331
Seattle, WA 98104-1404

 

DNS for Tor Exit Relaying

One of the major pieces of infrastructure run by Tor Exit Nodes is DNS. DNS is the system that translates human readable names, like emeraldonion.org, into IP addresses. In Tor, the exit node is the location where this translation takes place. As such, DNS has been recognized as one of the places where centralization or attacks could be performed that could affect the integrity of the Tor network. To serve our users well, we want to mitigate the risks of compromise and surveillance as we resolve names on behalf of our users. It’s these principles that direct how we structure our DNS resolution.

 

Emerald Onion currently uses pfSense, which uses Unbound for DNS. Per our architectural design, we run our own recursive DNS server, meaning we query up to the root name servers for DNS resolution, and avoid the cache any upstream ISPs offering us DNS resolvers. This also means we query the authoritative resolvers directly, minimizing the number of additional parties able to observe domain resolutions coming from our users.

General Settings:

  • We use DNS Resolver and disable the DNS Forwarder
  • Only bind the DNS listener to the NIC the Tor server is connected to and localhost.
  • Only bind the DNS outgoing interface to the NIC that carries our public IP. If you use BGP, do NOT bind DNS to the interface used to connect to BGP peers.
  • Enable DNSSEC support
  • Disable DNS Query Forwarding
  • We don’t use DHCP, leave DHCP Registration disabled
  • Same goes with Static DHCP

Custom options:

prefer-ip6: yes
hide-trustanchor: yes
harden-large-queries: yes
harden-algo-downgrade: yes
qname-minimisation-strict: yes
ignore-cd-flag: yes

The most important of these options is qname-minimization, which means that when we perform a resolution like www.emeraldonion.org, we ask the root name servers only for who controls .org, the Org name servers, who controls emeraldonion.org, and only ask emerald onion’s name servers for the IP of www.emeraldonion.org. This helps to protect against our traffic resolutions being swept into the various “passive DNS” feeds that have been commoditized around the network.

Of the other custom options, the bulk are related to DNSSEC security.

 

Advanced Settings:

  • Hide Identity
  • Hide Version
  • Use Prefetch Support
  • Use Prefetch DNS Keys
  • Harden DNSSEC data
  • Leave the tuning values alone for now (Things like cache size, buffers, queues, jostle, etc)
  • Log Level is 1, which is pretty low.
  • Leave the rest alone.

Hiding the identity and version helps prevent the leakage of information that could be used in attacks against us. Prefetch Support changes how the DNS server fetches DNS records. Without it, it fetches the DNS record at the time of the request. With Prefetch Support, it refreshes the DNS entries as each record’s TTL expires helping to further obfuscate requests and makes it harder for specific Tor request correlation attacks.

Access Lists:

We don’t use this, but if you want to work with the Access Lists, that should be fine, just keep it in mind when troubleshooting.

Legal Responses to Tor Abuse Complaints

Disclaimer: This is not legal advice due to the fact that Emerald Onion is not your lawyer. Further, it is based on laws and court cases in the United States and does not apply to most of the world. If you are planning on becoming an ISP dedicated to transiting Tor traffic, please seek out and maintain active support from your own lawyer.

One of Emerald Onion’s priorities when establishing was to work with professional legal services for specific language when communicating with other organizations negatively impacted by the anonymous Tor traffic that we transit. Another priority is to freely share our work, to help others who take on the cause.

If you value our not-for-profit work, like the information shared here, please keep in mind that we graciously accept donations.

There are three documents that we use for addressing Tor abuse complaints. The first one has been published on our site for over a month, our Legal FAQ. We use this document whenever someone contacts us about any issue that is not a DMCA issue, so it is used a lot. Keep in mind that Emerald Onion only operates full (not reduced) exit relays, so we get lots of infringement related complaints.

The two other documents (below) should be used based on where an abuse complaint is coming from. Most Tor operators are used to working with their upstream ISP directly. This is because these operators lease IP space from their upstream ISP. In this case, it is important to use the “Complaint Forwarded By Your ISP(s)”.

The other way that abuse complaints can come in is directly from third-parties, typically the ones who have recorded your IP address(es) as being the cause of perceived abuse. Emerald Onion went through the process of obtaining our own Autonomous System (AS) number, IPv6 and IPv4 scopes, and because of this, we receive all abuse complaints directly. Third-parties at the other end of the anonymous Tor traffic that we transit only see our contact information in the WHOIS for our IP space, and not the contact information of our upstream providers.

We encourage other Tor operators to replicate our work so that they can directly manage all abuse complaints. Doing so removes a tremendous amount of legal stress from upstream providers. When this is applicable to you, use the “Direct Complaint” document below.

Please also thank EFF and The Tor Project for their prior work in this area.

Complaint Forwarded By Your ISP(s)

I am writing on behalf of [Your Organization] in response to the notice copyright below.

[Your Organization] is a [Your State] State-based nonprofit corporation and encrypted transit internet service provider. I would like to assure you that [Your Organization] is neither infringing copyrights nor hosting the claimed infringing materials on behalf of users. The notice is likely based upon misunderstandings about the law and about some of the software we run.

As an initial matter, the faulty copyright notice was triggered by a routing program that we are running on our servers called Tor. The Tor network is a group of volunteer-operated routers on the Internet that helps people enhance their security, privacy, and safety. [Your Organization] only runs Tor traffic through our routers.

Tor bounces communications through routers on the network before sending those packets on to their final destinations. However, Tor tunnels the connections so that no router can learn both the source and destination of the packets, which protects users from nefarious snooping on network traffic. The result is that the final IP address that the recipient receives is not the IP address of the sender.

Tor is valuable because it promotes privacy and free expression around the world. It also protects users against hazards such as harassment, spam, identity theft, price discrimination, and even dissident surveillance by oppressive governments. To learn more about Tor, see https://www.torproject.org/.

[Your Organization] operates Tor exit routers with connectivity to the [Your Internet Exchange Point(s)]. Those routers are only a conduit of online traffic. We are thus protected by the Digital Millennium Copyright Act safe harbor provision of 512(a). That safe harbor also likely protects you, as our ISP, from liability arising from this complaint.

The DMCA creates four “safe harbors” that protect service providers from copyright liability for the acts of their users, provided that the ISPs fulfill certain requirements. 17 U.S.C. § 512. The DMCA’s requirements vary depending on the ISP’s role. You may be familiar with the “notice and takedown” provisions of section 512(c) of the DMCA, which applies to ISPs hosting content on behalf of users. However, those requirements do not apply when an ISP merely serves as a “conduit,” i.e., transmits, routes, or provides for connections through a system or network. The safe harbor of section 512(a) has different and less burdensome eligibility requirements for conduits. See Recording Indus. Ass’n of Am., Inc. v. Verizon Internet Servs., Inc., 351 F.3d 1229 (D.C. Cir. 2003); In re Charter Commc’ns, Inc., Subpoena Enforcement Matter, 393 F.3d 771 (8th Cir. 2005).

Under section 512(a), service providers like you are typically protected from damages for copyright infringement claims if you also maintain “a policy that provides for termination in appropriate circumstances of subscribers and account holders of the service provider’s system or network who are repeat infringers.” If you have and implement such a policy, and you otherwise qualify for the safe harbor, you should be free from fear of copyright damages.

As an organization committed to protecting the privacy of its customers, I hope you’ll agree that Tor is a valuable technology and will continue to support our effort to strengthen the Tor network.

Thank you for working with me on this matter. As a loyal subscriber, I appreciate your notifying me of this issue and hope that the protections of DMCA 512 put any concerns you may have to rest. If not, please contact me with any further questions.

Sincerely,

[Your Name], on behalf of [Your Organization]

 

Direct Complaint

I am writing on behalf of [Your Organization] in response to the notice copyright below.

[Your Organization] is a [Your State] State-based nonprofit corporation and encrypted transit internet service provider. I would like to assure you that [Your Organization] is neither infringing copyrights nor hosting the claimed infringing materials on behalf of users. The notice is likely based upon misunderstandings about the law and about some of the software we run.

As an initial matter, the faulty copyright notice was triggered by a routing program that we are running on our servers called Tor. The Tor network is a group of volunteer-operated routers on the Internet that helps people enhance their security, privacy, and safety. [Your Organization] only runs Tor traffic through our routers.

Tor bounces communications through routers on the network before sending those packets on to their final destinations. However, Tor tunnels the connections so that no router can learn both the source and destination of the packets, which protects users from nefarious snooping on network traffic. The result is that the final IP address that the recipient receives is not the IP address of the sender.

Tor is valuable because it promotes privacy and free expression around the world. It also protects users against hazards such as harassment, spam, identity theft, price discrimination, and even dissident surveillance by oppressive governments. To learn more about Tor, see https://www.torproject.org/.

As you may know, the DMCA creates four “safe harbors” that protect service providers from copyright liability for the acts of their users, provided that the ISPs fulfill certain requirements. 17 U.S.C. § 512. The DMCA’s requirements vary depending on the ISP’s role. You may be familiar with the “notice and takedown” provisions of section 512(c) of the DMCA, which applies to ISPs hosting content on behalf of users. However, those requirements do not apply when an ISP merely serves as a “conduit,” i.e., transmits, routes, or provides for connections through a system or network. The safe harbor of section 512(a) has different and less burdensome eligibility requirements for conduits. See Recording Indus. Ass’n of Am., Inc. v. Verizon Internet Servs., Inc., 351 F.3d 1229 (D.C. Cir. 2003); In re Charter Commc’ns, Inc., Subpoena Enforcement Matter, 393 F.3d 771 (8th Cir. 2005).

[Your Organization] operates Tor exit routers with connectivity to the [Your Internet Exchange Point(s)]. Because [Your Organization] serves as a conduit of online traffic, we fall within the section 512(a) safe harbor, and are not subject to the requirements of section 512(c).

Thank you for working with me on this matter. I hope that the protections of DMCA 512 put any concerns you may have to rest. If not, please contact me with any further questions.

Sincerely,

[Your Name], on behalf of [Your Organization]

 

Emerald Onion’s BGP Setup

This is a walk through of who our current peers are and our BGP setup.

Special thanks to DFRI, Paul English, Seattle Internet Exchange, and Theodore Baschak for your time and patience!

Current Peers

202 peers via the SIX Route Servers, 12 Direct Peers Peers via the SIX and 1 Transit Peer through the SIX.

6456   - Altopia Corporation
13335  - CloudFlare, Inc.
395823 - doof.net
36459  - Github
6939   - Hurricane Electric**
57695  - Misaka Network LLC
3856   - Packet Clearing House
42     - WoodyNet (Also Packet Clearing House)
23265  - Pocketinet Communications, Inc.
16652  - Riseup Networks
33108  - Seattle Internet Exchange*
64241  - Wobscale Technologies, LLC
10310  - Yahoo! Inc.

Updated 7/4/2018

* The Seattle Internet Exchange (SIX) peer is for Route Servers
** Hurricane Electric is our current transit provider.

To see a list of all peers through the route servers:

BGP Setup

Since we currently use pfSense, we use openbgpd to peer with other Autonomous Systems.

In order to accomplish this, there are a few pre-requisites:

  1. An AS Number (ASN). Check out the list of Regional Internet Registries (RIR) for your respective geographical location on getting your ASN and Direct Allocation of IP Addresses (IPv6 & IPv4). They are listed at the bottom in the External Resources section of this page.
  2. If peering with an Internet Exchange Point (IXP) a dedicated IP address from them in order to peer (Both IPv6 & IPv4).
  3. Install the openbgpd package in pfSense (System > Package Manager > Available Packages) and then enter OpenBGPD.
  4. Submit a Letter of Agency (LOA) to your transit provider so they can announce your ASN thus IP space upstream.
  5. When switching from a typical router config to that of a BGP router, there are some fundamental changes in architecture that are required. Take a look at our Conversion Article here: https://emeraldonion.org/eo-pfsense-conversion-plan/

A fundamental aspect to this setup is touched on in the conversion plan linked in step 5. It is important to understand that a typical router setup is that the WAN links have default gateways but when setting up or switching to BGP connections, Default Gateways are not used and must be removed from the NIC config. If you want your transit provider to be your default route, you ask them to advertise that route to you and then through BGP you will get the 0.0.0.0 route. In our case, our transit provider is WowRack (AS23033) and they advertise the default route to us. The other ASNs that we peer with do not and it is BGP’s job to select the correct route based on AS length.

We found that after installing the openbgpd package in pfSense, it is best to just use the raw config tab (Services > OpenBGPD > Raw config). The issue we ran into is that after filling out the wizard, we needed to make some changes. Doing so through the wizard didn’t update the raw config which is what the service actually looks at (bgpd.conf). So, now we just manage it through the raw config.

 

Our BGP Config

At a high level, there are 3 major parts to the config:

Router Config

Such as ASN, Router ID, Network Info and Options (Like fib update and holdtime).

Groups and Neighbors

This will have a bunch of groups with neighbors in them. It can also have groups that contains two Neighbors. A group being a single AS and Neighbors being a couple of routers that Neighbor has (usually for redundancy).

We highly recommend peering with your local Internet Exchange’s (IX) route servers. This is an easy way to peer with a bunch of ASNs without having to setup direct peering. Route servers are however not a substitute for direct peering. When doing this, make sure in the bgpd.conf in the neighbor section of the group to tell bgpd not to enforce the neighbor as using “enforce neighbor-as no” so that it will accept routes from ASNs that aren’t the same as the route servers’ peering ASN.

Filtering Rules

This is how we allow or deny routes to come through from our peers. First we block everything, then we allow our peers, then we block specific networks like Martians (Such as RFC1918, etc).

We recently made some changes to this section to help protect against some poor practices seen in BGP configs. One thing is to append “inet prefixlen 8 – 24” for IPv4 and “inet6 prefixlen 16 – 48” for IPv6 to the end of the allow from and allow to statements. This states that we will only accept networks with a size of /8 to /24 (IPv4) and /16 to /48 (IPv6).

And we also made some updates to the bogon network list per the OpenBGPD standard config. These networks aren’t meant for Internet traffic so we filter them out.

bgpd.conf

AS 396507

fib-update yes
holdtime 90

router-id 206.81.81.158

# IPv4 network
network 23.129.64.0/24
# IPv6 network
network 2620:18C::/36

#### IPv4 neighbors ####
group "AS-WOWRACK-Transit-v4" {
	remote-as 23033
	neighbor 216.176.186.129 {
		descr "WOW_trans_rs1v4"
		announce self
		local-address 216.176.186.130
		max-prefix 1000000
}
}
group "AS-SIXRSv4" {
	remote-as 33108
	neighbor 206.81.80.2 {
		descr "SIXRS_rs2v4"
		announce self
		local-address 206.81.81.158
		enforce neighbor-as no
		max-prefix 200000
}
	neighbor 206.81.80.3 {
		descr "SIXRS_rs3v4"
		announce self
		local-address 206.81.81.158
		enforce neighbor-as no
		max-prefix 200000
}
}
group "AS-HURRICANEv4" {
	remote-as 6939
	neighbor 206.81.80.40 {
		descr "HE_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 152000
}
}
group "AS-ALTOPIAv4" {
	remote-as 6456
	neighbor 206.81.80.10 {
		descr "ALT_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 20 restart 30
}
	neighbor 206.81.81.41 {
		descr "ALT_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 20 restart 30
}
}
group "AS-POCKETINETv4" {
	remote-as 23265
	neighbor 206.81.80.88 {
		descr "POK_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 600
}
}
group "AS-DOOFv4" {
	remote-as 395823
	neighbor 206.81.81.125 {
		descr "DOOF_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 5
}
}
group "AS-PCHv4" {
	remote-as 3856
	neighbor 206.81.80.81 {
		descr "PCH_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 600
}
}
group "AS-PCHWNv4" {
	remote-as 42
	neighbor 206.81.80.80 {
		descr "PCHWN_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 600
}
}
group "AS-WOBv4" {
	remote-as 64241
	neighbor 206.81.81.87 {
		descr "WOB_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 5
}
}
group "AS-GOOGv4" {
	remote-as 15169
	neighbor 206.81.80.17 {
		descr "GOOG_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 15000
}
}
group "AS-MISAKAv4" {
	remote-as 57695
	neighbor 206.81.81.161 {
		descr "MISAKA_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 200
}
}
group "AS-RISUPv4" {
	remote-as 16652
	neighbor 206.81.81.74 {
		descr "RISUP_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 20
}
}
group "AS-AKAMAIv4" {
	remote-as 20940
	neighbor 206.81.80.113 {
		descr "AKAMAI_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 200
}
}
group "AS-CoSITv4" {
	remote-as 3401
	neighbor 206.81.80.202 {
		descr "CoSIT_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 10
}
}
group "AS-CLDFLRv4" {
	remote-as 13335
	neighbor 206.81.81.10 {
		descr "CLDFLR_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 1000
}
}
group "AS-DYNv4" {
	remote-as 33517
	neighbor 206.81.81.121 {
		descr "DYN_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 400
}
}
group "AS-FCBKv4" {
	remote-as 32934
	neighbor 206.81.80.181 {
		descr "FCBK_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 200
}
	neighbor 206.81.80.211 {
		descr "FCBK_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 200
}
}
group "AS-GITHUBv4" {
	remote-as 36459
	neighbor 206.81.81.89 {
		descr "GITHUB_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 100
}
	neighbor 206.81.81.90 {
		descr "GITHUB_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 100
}
}
group "AS-MSFTv4" {
	remote-as 8075
	neighbor 206.81.80.30 {
		descr "MSFT_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 2000
}
	neighbor 206.81.80.68 {
		descr "MSFT_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 2000
}
}
group "AS-OpenDNSv4" {
	remote-as 36692
	neighbor 206.81.80.53 {
		descr "OpenDNS_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 200
}
}
group "AS-SPLv4" {
	remote-as 21525
	neighbor 206.81.80.196 {
		descr "SPL_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 10
}
}
group "AS-TWITTERv4" {
	remote-as 13414
	neighbor 206.81.81.31 {
		descr "TWITTER_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 200
}
}
group "AS-VRISIGNv4" {
	remote-as 7342
	neighbor 206.81.80.133 {
		descr "VRISIGN_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 600
}
}
group "AS-YAHOOv4" {
	remote-as 10310
	neighbor 206.81.80.98 {
		descr "YAHOO_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 2000
}
	neighbor 206.81.81.50 {
		descr "YAHOO_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 2000
}
}
group "AS-INTEGRAv4" {
	remote-as 7385
	neighbor 206.81.80.102 {
		descr "INTEGRA_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 2000
}
}
group "AS-PNWGPv4" {
	remote-as 101
	neighbor 206.81.80.84 {
		descr "PNWGP_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 500
}
}
group "AS-WAVEv4" {
	remote-as 11404
	neighbor 206.81.80.56 {
		descr "WAVE_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 6000
}
}
group "AS-AMAZONv4" {
	remote-as 16509
	neighbor 206.81.80.147 {
		descr "AMAZON_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 4000
}
	neighbor 206.81.80.248 {
		descr "AMAZON_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 4000
}
}
group "AS-SYMTECv4" {
	remote-as 27471
	neighbor 206.81.81.169 {
		descr "SYMTEC_rs1v4"
		announce self
		local-address 206.81.81.158
		max-prefix 40
}
	neighbor 206.81.81.170 {
		descr "SYMTEC_rs2v4"
		announce self
		local-address 206.81.81.158
		max-prefix 40
}
}

#### IPv6 neighbors ####
group "AS-WOWRACK-Transit-v6" {
	remote-as 23033
	neighbor 2607:F8F8:2F0:811:2::1 {
		descr "WOW_trans_rs1v6"
		announce self
		local-address 2607:F8F8:2F0:811:2::2
		max-prefix 100000
}
}
group "AS-SIXRSv6" {
	remote-as 33108
	neighbor 2001:504:16::2 {
		descr "SIXRS_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		enforce neighbor-as no
		max-prefix 60000
}
	neighbor 2001:504:16::3 {
		descr "SIXRS_rs3v6"
		announce self
		local-address 2001:504:16::6:cdb
		enforce neighbor-as no
		max-prefix 60000
}
}
group "AS-HURRICANEv6" {
	remote-as 6939
	neighbor 2001:504:16::1b1b {
		descr "HE_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 41000
}
}
group "AS-ALTOPIAv6" {
	remote-as 6456
	neighbor 2001:504:16::1938 {
		descr "ALT_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 20 restart 30
}
	neighbor 2001:504:16::297:0:1938 {
		descr "ALT_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 20 restart 30
}
}
group "AS-POCKETINETv6" {
	remote-as 23265
	neighbor 2001:504:16::5ae1 {
		descr "POK_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 600
}
}
group "AS-DOOFv6" {
	remote-as 395823
	neighbor 2001:504:16::6:a2f {
		descr "DOOF_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 5
}
}
group "AS-PCHv6" {
	remote-as 3856
	neighbor 2001:504:16::f10 {
		descr "PCH_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 600
}
}
group "AS-PCHWNv6" {
	remote-as 42
	neighbor 2001:504:16::2a {
		descr "PCHWN_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 600
}
}
group "AS-WOBv6" {
	remote-as 64241
	neighbor 2001:504:16::faf1 {
		descr "WOB_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 5
}
}
group "AS-GOOGv6" {
	remote-as 15169
	neighbor 2001:504:16::3b41 {
		descr "GOOG_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 750
}
}
group "AS-MISAKAv6" {
	remote-as 57695
	neighbor 2001:504:16::e15f {
		descr "MISAKA_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 150
}
}
group "AS-RISUPv6" {
	remote-as 16652
	neighbor 2001:504:16::410c {
		descr "RISUP_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 10
}
}
group "AS-AKAMAIv6" {
	remote-as 20940
	neighbor 2001:504:16::51cc {
		descr "AKAMAI_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 40
}
}
group "AS-CLDFLRv6" {
	remote-as 13335
	neighbor 2001:504:16::3417 {
		descr "CLDFLR_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 200
}
}
group "AS-DYNv6" {
	remote-as 33517
	neighbor 2001:504:16::82ed {
		descr "DYN_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 200
}
}
group "AS-FCBKv6" {
	remote-as 32934
	neighbor 2001:504:16::80a6 {
		descr "FCBK_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 200
}
	neighbor 2001:504:16::211:0:80a6 {
		descr "FCBK_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 200
}
}
group "AS-GITHUBv6" {
	remote-as 36459
	neighbor 2001:504:16::8e6b {
		descr "GITHUB_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 20
}
	neighbor 2001:504:16::346:0:8e6b {
		descr "GITHUB_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 20
}
}
group "AS-MSFTv6" {
	remote-as 8075
	neighbor 2001:504:16::1f8b {
		descr "MSFT_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 500
}
	neighbor 2001:504:16::68:0:1f8b {
		descr "MSFT_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 500
}
}
group "AS-OpenDNSv6" {
	remote-as 36692
	neighbor 2001:504:16::8f54 {
		descr "OpenDNS_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 40
}
}
group "AS-SPLv6" {
	remote-as 21525
	neighbor 2001:504:16::5415 {
		descr "SPL_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 10
}
}
group "AS-TWITTERv6" {
	remote-as 13414
	neighbor 2001:504:16::3466 {
		descr "TWITTER_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 10
}
}
group "AS-VRISIGNv6" {
	remote-as 7342
	neighbor 2001:504:16::1cae {
		descr "VRISIGN_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 100
}
}
group "AS-YAHOOv6" {
	remote-as 10310
	neighbor 2001:504:16::2846 {
		descr "YAHOO_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 200
}
	neighbor 2001:504:16::306:0:2846 {
		descr "YAHOO_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 200
}
}
group "AS-INTEGRAv6" {
	remote-as 7385
	neighbor 2001:504:16::1cd9 {
		descr "INTEGRA_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 100
}
}
group "AS-PNWGPv6" {
	remote-as 101
	neighbor 2001:504:16::65 {
		descr "PNWGP_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 20
}
}
group "AS-WAVEv6" {
	remote-as 11404
	neighbor 2001:504:16::2c8c {
		descr "WAVE_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 500
}
}
group "AS-AMAZONv6" {
	remote-as 16509
	neighbor 2001:504:16::407d {
		descr "AMAZON_rs1v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 1000
}
	neighbor 2001:504:16::248:0:407d {
		descr "AMAZON_rs2v6"
		announce self
		local-address 2001:504:16::6:cdb
		max-prefix 1000
}
}

#### Filtering Rules ####

deny from any
deny to any

# https://www.arin.net/announcements/2014/20140130.html
# This block will be subject to a minimum size allocation of /28 and a
# maximum size allocation of /24. ARIN should use sparse allocation when
# possible within that /10 block.
allow from any prefix 23.128.0.0/10 prefixlen 24 - 28   # ARIN IPv6 transition

## IPv4 ##
# WOW_trans_rs1v4
allow from 216.176.186.129
allow to 216.176.186.129
# SIXRS_rs2v4
allow from 206.81.80.2 inet prefixlen 8 - 24
allow to 206.81.80.2 inet prefixlen 8 - 24
# SIXRS_rs3v4
allow from 206.81.80.3 inet prefixlen 8 - 24
allow to 206.81.80.3 inet prefixlen 8 - 24
# HE_rs1v4
allow from 206.81.80.40
allow to 206.81.80.40
# ALT_rs1v4
allow from 206.81.80.10 inet prefixlen 8 - 24
allow to 206.81.80.10 inet prefixlen 8 - 24
# ALT_rs2v4
allow from 206.81.81.41 inet prefixlen 8 - 24
allow to 206.81.81.41 inet prefixlen 8 - 24
# POK_rs1v4
allow from 206.81.80.88 inet prefixlen 8 - 24
allow to 206.81.80.88 inet prefixlen 8 - 24
# DOOF_rs1v4
allow from 206.81.81.125 inet prefixlen 8 - 24
allow to 206.81.81.125 inet prefixlen 8 - 24
# PCH_rs1v4
allow from 206.81.80.81 inet prefixlen 8 - 24
allow to 206.81.80.81 inet prefixlen 8 - 24
# PCHWN_rs1v4
allow from 206.81.80.80 inet prefixlen 8 - 24
allow to 206.81.80.80 inet prefixlen 8 - 24
# WOB_rs1v4
allow from 206.81.81.87 inet prefixlen 8 - 24
allow to 206.81.81.87 inet prefixlen 8 - 24
# GOOG_rs1v4
allow from 206.81.80.17
allow to 206.81.80.17
# MISAKA_rs1v4
allow from 206.81.81.161 inet prefixlen 8 - 24
allow to 206.81.81.161 inet prefixlen 8 - 24
# RISUP_rs1v4
allow from 206.81.81.74 inet prefixlen 8 - 24
allow to 206.81.81.74 inet prefixlen 8 - 24
# AKAMAI_rs1v4
allow from 206.81.80.113 inet prefixlen 8 - 24
allow to 206.81.80.113 inet prefixlen 8 - 24
# CoSIT_rs1v4
allow from 206.81.80.202 inet prefixlen 8 - 24
allow to 206.81.80.202 inet prefixlen 8 - 24
# CLDFLR_rs1v4
allow from 206.81.81.10 inet prefixlen 8 - 24
allow to 206.81.81.10 inet prefixlen 8 - 24
# DYN_rs1v4
allow from 206.81.81.121 inet prefixlen 8 - 24
allow to 206.81.81.121 inet prefixlen 8 - 24
# FCBK_rs1v4
allow from 206.81.80.181 inet prefixlen 8 - 24
allow to 206.81.80.181 inet prefixlen 8 - 24
# FCBK_rs2v4
allow from 206.81.80.211 inet prefixlen 8 - 24
allow to 206.81.80.211 inet prefixlen 8 - 24
# GITHUB_rs1v4
allow from 206.81.81.89 inet prefixlen 8 - 24
allow to 206.81.81.89 inet prefixlen 8 - 24
# GITHUB_rs2v4
allow from 206.81.81.90 inet prefixlen 8 - 24
allow to 206.81.81.90 inet prefixlen 8 - 24
# MSFT_rs1v4
allow from 206.81.80.30 inet prefixlen 8 - 24
allow to 206.81.80.30 inet prefixlen 8 - 24
# MSFT_rs2v4
allow from 206.81.80.68 inet prefixlen 8 - 24
allow to 206.81.80.68 inet prefixlen 8 - 24
# OpenDNS_rs1v4
allow from 206.81.80.53 inet prefixlen 8 - 24
allow to 206.81.80.53 inet prefixlen 8 - 24
# SPL_rs1v4
allow from 206.81.80.196 inet prefixlen 8 - 24
allow to 206.81.80.196 inet prefixlen 8 - 24
# TWITTER_rs1v4
allow from 206.81.81.31 inet prefixlen 8 - 24
allow to 206.81.81.31 inet prefixlen 8 - 24
# VRISIGN_rs1v4
allow from 206.81.80.133 inet prefixlen 8 - 24
allow to 206.81.80.133 inet prefixlen 8 - 24
# YAHOO_rs1v4
allow from 206.81.80.98 inet prefixlen 8 - 24
allow to 206.81.80.98 inet prefixlen 8 - 24
# YAHOO_rs2v4
allow from 206.81.81.50 inet prefixlen 8 - 24
allow to 206.81.81.50 inet prefixlen 8 - 24
# INTEGRA_rs1v4
allow from 206.81.80.102 inet prefixlen 8 - 24
allow to 206.81.80.102 inet prefixlen 8 - 24
# PNWGP_rs1v4
allow from 206.81.80.84 inet prefixlen 8 - 24
allow to 206.81.80.84 inet prefixlen 8 - 24
# WAVE_rs1v4
allow from 206.81.80.56 inet prefixlen 8 - 24
allow to 206.81.80.56 inet prefixlen 8 - 24
# AMAZON_rs1v4
allow from 206.81.80.147 inet prefixlen 8 - 24
allow to 206.81.80.147 inet prefixlen 8 - 24
# AMAZON_rs2v4
allow from 206.81.80.248 inet prefixlen 8 - 24
allow to 206.81.80.248 inet prefixlen 8 - 24
# SYMTEC_rs1v4
allow from 206.81.81.169 inet prefixlen 8 - 24
allow to 206.81.81.169 inet prefixlen 8 - 24
# SYMTEC_rs2v4
allow from 206.81.81.170 inet prefixlen 8 - 24
allow to 206.81.81.170 inet prefixlen 8 - 24

## IPv6 ##
# WOW_trans_rs1v6
allow from 2607:F8F8:2F0:811:2::1
allow to 2607:F8F8:2F0:811:2::1
# SIXRS_rs2v6
allow from 2001:504:16::2 inet6 prefixlen 16 - 48
allow to 2001:504:16::2 inet6 prefixlen 16 - 48
# SIXRS_rs3v6
allow from 2001:504:16::3 inet6 prefixlen 16 - 48
allow to 2001:504:16::3 inet6 prefixlen 16 - 48
# HE_rs1v6
allow from 2001:504:16::1b1b
allow to 2001:504:16::1b1b
# ALT_rs1v6
allow from 2001:504:16::1938 inet6 prefixlen 16 - 48
allow to 2001:504:16::1938 inet6 prefixlen 16 - 48
# ALT_rs2v6
allow from 2001:504:16::297:0:1938 inet6 prefixlen 16 - 48
allow to 2001:504:16::297:0:1938 inet6 prefixlen 16 - 48
# POK_rs1v6
allow from 2001:504:16::5ae1 inet6 prefixlen 16 - 48
allow to 2001:504:16::5ae1 inet6 prefixlen 16 - 48
# DOOF_rs1v6
allow from 2001:504:16::6:a2f inet6 prefixlen 16 - 48
allow to 2001:504:16::6:a2f inet6 prefixlen 16 - 48
# PCH_rs1v6
allow from 2001:504:16::f10 inet6 prefixlen 16 - 48
allow to 2001:504:16::f10 inet6 prefixlen 16 - 48
# PCHWN_rs1v6
allow from 2001:504:16::2a inet6 prefixlen 16 - 48
allow to 2001:504:16::2a inet6 prefixlen 16 - 48
# WOB_rs1v6
allow from 2001:504:16::faf1 inet6 prefixlen 16 - 48
allow to 2001:504:16::faf1 inet6 prefixlen 16 - 48
# GOOG_rs1v6
allow from 2001:504:16::3b41
allow to 2001:504:16::3b41
# MISAKA_rs1v6
allow from 2001:504:16::e15f inet6 prefixlen 16 - 48
allow to 2001:504:16::e15f inet6 prefixlen 16 - 48
# RISUP_rs1v6
allow from 2001:504:16::410c inet6 prefixlen 16 - 48
allow to 2001:504:16::410c inet6 prefixlen 16 - 48
# AKAMAI_rs1v6
allow from 2001:504:16::51cc inet6 prefixlen 16 - 48
allow to 2001:504:16::51cc inet6 prefixlen 16 - 48
# CLDFLR_rs1v6
allow from 2001:504:16::3417 inet6 prefixlen 16 - 48
allow to 2001:504:16::3417 inet6 prefixlen 16 - 48
# DYN_rs1v6
allow from 2001:504:16::82ed inet6 prefixlen 16 - 48
allow to 2001:504:16::82ed inet6 prefixlen 16 - 48
# FCBK_rs1v6
allow from 2001:504:16::80a6 inet6 prefixlen 16 - 48
allow to 2001:504:16::80a6 inet6 prefixlen 16 - 48
# FCBK_rs2v6
allow from 2001:504:16::211:0:80a6 inet6 prefixlen 16 - 48
allow to 2001:504:16::211:0:80a6 inet6 prefixlen 16 - 48
# GITHUB_rs1v6
allow from 2001:504:16::8e6b inet6 prefixlen 16 - 48
allow to 2001:504:16::8e6b inet6 prefixlen 16 - 48
# GITHUB_rs2v6
allow from 2001:504:16::346:0:8e6b inet6 prefixlen 16 - 48
allow to 2001:504:16::346:0:8e6b inet6 prefixlen 16 - 48
# MSFT_rs1v6
allow from 2001:504:16::1f8b inet6 prefixlen 16 - 48
allow to 2001:504:16::1f8b inet6 prefixlen 16 - 48
# MSFT_rs2v6
allow from 2001:504:16::68:0:1f8b inet6 prefixlen 16 - 48
allow to 2001:504:16::68:0:1f8b inet6 prefixlen 16 - 48
# OpenDNS_rs1v6
allow from 2001:504:16::8f54 inet6 prefixlen 16 - 48
allow to 2001:504:16::8f54 inet6 prefixlen 16 - 48
# SPL_rs1v6
allow from 2001:504:16::5415 inet6 prefixlen 16 - 48
allow to 2001:504:16::5415 inet6 prefixlen 16 - 48
# TWITTER_rs1v6
allow from 2001:504:16::3466 inet6 prefixlen 16 - 48
allow to 2001:504:16::3466 inet6 prefixlen 16 - 48
# VRISIGN_rs1v6
allow from 2001:504:16::1cae inet6 prefixlen 16 - 48
allow to 2001:504:16::1cae inet6 prefixlen 16 - 48
# YAHOO_rs1v6
allow from 2001:504:16::2846 inet6 prefixlen 16 - 48
allow to 2001:504:16::2846 inet6 prefixlen 16 - 48
# YAHOO_rs2v6
allow from 2001:504:16::306:0:2846 inet6 prefixlen 16 - 48
allow to 2001:504:16::306:0:2846 inet6 prefixlen 16 - 48
# INTEGRA_rs1v6
allow from 2001:504:16::1cd9 inet6 prefixlen 16 - 48
allow to 2001:504:16::1cd9 inet6 prefixlen 16 - 48
# PNWGP_rs1v6
allow from 2001:504:16::65 inet6 prefixlen 16 - 48
allow to 2001:504:16::65 inet6 prefixlen 16 - 48
# WAVE_rs1v6
allow from 2001:504:16::2c8c inet6 prefixlen 16 - 48
allow to 2001:504:16::2c8c inet6 prefixlen 16 - 48
# AMAZON_rs1v6
allow from 2001:504:16::407d inet6 prefixlen 16 - 48
allow to 2001:504:16::407d inet6 prefixlen 16 - 48
# AMAZON_rs2v6
allow from 2001:504:16::248:0:407d inet6 prefixlen 16 - 48
allow to 2001:504:16::248:0:407d inet6 prefixlen 16 - 48

# filter bogus networks according to RFC5735
deny from any prefix 0.0.0.0/8 prefixlen >= 8           # 'this' network [RFC1122]
deny from any prefix 10.0.0.0/8 prefixlen >= 8          # private space [RFC1918]
deny from any prefix 100.64.0.0/10 prefixlen >= 10      # CGN Shared [RFC6598]
deny from any prefix 127.0.0.0/8 prefixlen >= 8         # localhost [RFC1122]
deny from any prefix 169.254.0.0/16 prefixlen >= 16     # link local [RFC3927]
deny from any prefix 172.16.0.0/12 prefixlen >= 12      # private space [RFC1918]
deny from any prefix 192.0.2.0/24 prefixlen >= 24       # TEST-NET-1 [RFC5737]
deny from any prefix 192.168.0.0/16 prefixlen >= 16     # private space [RFC1918]
deny from any prefix 198.18.0.0/15 prefixlen >= 15      # benchmarking [RFC2544]
deny from any prefix 198.51.100.0/24 prefixlen >= 24    # TEST-NET-2 [RFC5737]
deny from any prefix 203.0.113.0/24 prefixlen >= 24     # TEST-NET-3 [RFC5737]
deny from any prefix 224.0.0.0/4 prefixlen >= 4         # multicast
deny from any prefix 240.0.0.0/4 prefixlen >= 4         # reserved

# filter bogus IPv6 networks according to IANA
deny from any prefix ::/8 prefixlen >= 8
deny from any prefix 0100::/64 prefixlen >= 64          # Discard-Only [RFC6666]
deny from any prefix 2001:2::/48 prefixlen >= 48        # BMWG [RFC5180]
deny from any prefix 2001:10::/28 prefixlen >= 28       # ORCHID [RFC4843]
deny from any prefix 2001:db8::/32 prefixlen >= 32      # docu range [RFC3849]
deny from any prefix 3ffe::/16 prefixlen >= 16          # old 6bone
deny from any prefix fc00::/7 prefixlen >= 7            # unique local unicast
deny from any prefix fe80::/10 prefixlen >= 10          # link local unicast
deny from any prefix fec0::/10 prefixlen >= 10          # old site local unicast
deny from any prefix ff00::/8 prefixlen >= 8            # multicast

Updated 9/5/2017

We’ll update this as we make changes.

External Resources

Here are a few references we leveraged when building our config:

RIRs:

African Network Information Center (AFRINIC) for Africa
https://www.afrinic.net/

American Registry for Internet Numbers (ARIN) for the United States, Canada, several parts of the Caribbean region, and Antarctica.
https://www.arin.net/

Asia-Pacific Network Information Centre (APNIC) for Asia, Australia, New Zealand, and neighboring countries
https://www.apnic.net/

Latin America and Caribbean Network Information Centre (LACNIC) for Latin America and parts of the Caribbean region
https://www.lacnic.net/

Réseaux IP Européens Network Coordination Centre (RIPE) for Europe, Russia, the Middle East, and Central Asia
https://www.ripe.net/

DNSSEC is now fully implemented for our forward and reverse lookup zones

Last month (July 2017) we moved our DNS zone management to the Google Cloud Platform since our domains were already registered with Google. After applying for the DNSSEC alpha, we were granted access and turned on DNSSEC for all three of our forward (domain) and reverse (IPv6 and IPv4 scopes) lookup zones. Google’s alpha products come with no SLA, so we took a risk implementing DNSSEC through Google.

Turning on DNSSEC was as easy flipping a switch in the control panel. The last part is adding the DS entries at the Registrar.

In the upper-right hand corner of Zone Details is Registrar Setup. This is where we got our DS entry information.

This DS information translates to a specific Key Tag, Algorithm, Digest Type, and Digest that needs to go into Google Domains (the actual Registrar).

This completed the domain setup. Now we needed to configure DNSSEC for our reverse lookup zones. Because they are direct allocations from ARIN, we needed to copy over the DS details over to ARIN.

View and Manage Your Networks > View & Manage Network (for both our IPv6 and IPv4 scopes) > Actions > Manage Reverse DNS > (select the delegation) > Modify DS Records

String (for our IPv6) parsed:

3600 DS 46756 8 2 5396635C919BAF34F24011FAB2DE251630AE2B8C17F1B69D05BCFDD603510014

String (for our IPv4) parsed:

3600 DS 40286 8 2 54686118794BD67CC76295F3D7F1C269D70EB5646F5DA130CC590AE14B33935F

This completed the ARIN DNSSEC configuration. While Google provided a quick DNS update for validation, ARIN took over 12 hours.

Internet Exchange Points in the United States

Emerald Onion is researching IXPs in the U.S.A. in order to identify areas of priority as it concerns increasing global Tor network capacity by way of putting Tor routers directly on these highly interconnected networks. Putting Tor exit routers in IXPs, for example, may reduce network latency to end points. It may also reduce network hops, potentially minimizing the possibility of third-party surveillance. Emerald Onion envisions a future where the Tor network is composed of much larger and more stable network operators, globally.

Questions

  1. Are there any Tor routers connected to any United States-based IXPs? If so, which ones and who operates them?
  2. Is this IXP friendly to Tor?
  3. What is the organizational structure of this IXP? Such as corporate-run or community-driven, etc.
  4. What qualities of an IXP should impact how meaningful it would be for the Tor network?
    • Number of participants?
    • Access to specific participants?
    • Nonprofit?
    • Community driven?
    • Affordability?
    • Geolocation?
    • Prohibits network surveillance?

A top 20 list of cities to focus on for Tor development?

  1. Chicago, IL has at least 12 IXPs
  2. New York City, NY has at least 9 IXPs (and has Calyx Institute)
  3. Dallas, TX has at least 6 IXPs
  4. Los Angeles, CA has at least 6 IXPs
  5. Miami, FL has at least 6 IXPs
  6. Seattle, WA has at least 5 IXPs (and has Riseup and Emerald Onion)
  7. San Jose, CA has at least 5 IXPs
  8. Phoenix, AZ has at least 5 IXPs
  9. Ashburn, VA has at least 3 IXPs
  10. Reston, VA has at least 3 IXPs
  11. Boston, MA has at least 3 IXPs
  12. Atlanta, GA has at least 3 IXPs
  13. Portland, OR has at least 3 IXPs
  14. Honolulu, HI has at least 2 IXPs
  15. Denver, CO has at least 2 IXPs
  16. Vienna, VA has at least 2 IXPs
  17. Palo Alto, CA has at least 1 IXP
  18. Salt Lake City, UT has at least 1 IXP (and has XMission)
  19. Minneapolis, MN has at least 1 IXP
  20. Detroit, MI has at least 1 IXP

IXPs in the United States

Ashburn, VA

    1. Equinix Ashburn Exchange (Equinix Ashburn)
    2. LINX Northern Virginia (LINX)
    3. MAE East

Ashland, VA

    1. Richmond Virginia Internet Exchange (RVA-IX)

Atlanta, GA

    1. Digital Realty / Telx Internet Exchange (TIE)
    2. Equinix Internet Exchange Atlanta (Equinix Atlanta)
    3. Southeast Network Access Point (SNAP)

Austin, TX

    1. CyrusOne Internet Exchange (CyrusOne IX)

Billings, MT

    1. Yellowstone Regional Internet eXchange (YRIX)

Boston, MA

    1. Boston Internet Exchange
    2. Massachusetts eXchange Point (MXP)
    3. CoreSite – Any2 Boston

Buffalo, NY

    1. Buffalo Niagara International Internet Exchange (BNIIX)

Chicago, IL

    1. AMS-IX Chicago
    2. CyrusOne Internet Exchange (CyrusOne IX)
    3. Equinix Chicago Exchange (Equinix Chicago)
    4. Greater Chicago International Internet Exchange (GCIIX)
    5. United IX – Chicago (ChIX)
    6. CoreSite – Any2 Chicago
    7. MAE Central

Columbus, OH

    1. OhioIX

Dallas, TX

    1. CyrusOne Internet Exchange (CyrusOne IX)
    2. DE-CIX, the Dallas Internet Exchange (DE-CIX Dallas)
    3. Digital Realty / Telx Internet Exchange (TIE)
    4. Equinix Dallas Exchange (Equinix Dallas)
    5. MAE Central
    6. Megaport MegaIX Dallas (MegaIX Dallas)

Denver, CO

    1. CoreSite – Any2 Denver
    2. Interconnection eXchange Denver (IX-Denver)

Detroit, MI

    1. Detroit Internet Exchange (DET-IX)

Duluth, NM

    1. Twin Ports Internet Exchange (TP-IX)

Gillette, WY

    1. BigHorn Fiber Internet Exchang (BFIX)

Hagåtña, Guam

    1. Guam Internet Exchange (GU-IX)

Honolulu, HI

    1. DRFortress Exchange (DRF IX)
    2. Hawaii Internet eXchange (HIX)

Houston, TX

    1. CyrusOne Internet Exchange (CyrusOne IX)

Indianapolis, IN

    1. Midwest Internet Exchange (MidWest-IX – Indy)

Jacksonville, FL

    1. Jacksonville Internet Exchange (JXIX)

Kansas City, MO

    1. Kansas City Internet eXchange (KCIX)

Los Angeles, CA

    1. CENIC International Internet eXchange (CIIX)
    2. Equinix Los Angeles Exchange (Equinix Los Angeles)
    3. Los Angeles International Internet eXchange (LAIIX)
    4. MAE West
    5. Pacific Wave Exchange in Los Angeles and Seattle (PacificWave)
    6. CoreSite – Any2 California

Madison, WI

    1. Madison Internet Exchange (MadIX)

Manassas, VA

    1. LINX Northern Virginia (LINX)

Medford, OR

    1. Southern Oregon Access Exchange (SOAX)

Miami, FL

    1. Equinix Internet Exchange Miami (Equinix Miami)
    2. MAE East
    3. Miami Internet Exchange (MiamiIX)
    4. NAP of the Americas (NOTA)
    5. The South Florida Internet Exchange (FL-IX)
    6. CoreSite – Any2 Miami

Milwaukee, WI

    1. The Milwaukee IX (MKE-IX)

Minneapolis, MN

    1. Midwest Internet Cooperative Exchange (MICE)

Moffett Field, CA

    1. NGIX West

Nashville, TN

    1. Nashville Internet Exchange (NashIX)

New York, NY

    1. AMS-IX New York (AMS-IX NY)
    2. Big Apple Peering Exchange (BigApe)
    3. Digital Realty / Telx Internet Exchange (TIE)
    4. Equinix Internet Exchange New York (Equinix New York)
    5. Free NYIIX Alternative (NYCX)
    6. New York, NY – (CoreSite – Any2 New York)
    7. DE-CIX, the New York / New Jersey Internet Exchange (DE-CIX New York)
    8. New York International Internet eXchange (NYIIX)
    9. MAE East

Omaha, NE

    1. Omaha Internet Exchange (OmahaIX)

Palo Alto, CA

    1. Equinix Internet Exchange Palo Alto (Equinix Palo Alto)

Philadelphia, PA

    1. Philadelphia Internet Exchange (PHILAIX)

Phoenix, AZ

    1. Arizona Internet Exchange (AZIX)
    2. Digital Realty / Telx Internet Exchange (TIE)
    3. Phoenix Internet Exchange, LLC (PHX-IX)
    4. Phoenix IX
    5. CyrusOne Internet Exchange (CyrusOne IX)

Portland, OR

    1. Central Oregon Internet eXchange (COIX)
    2. Northwest Access Exchange, Inc. (NWAX)
    3. Oregon Internet Exchange (OIX)

Reno, NV

    1. Tahoe Internet Exchange (TahoeIX)

Reston, VA

    1. LINX Northern Virginia (LINX)
    2. MAE East
    3. CoreSite – Any2 NorthEast

Saint George, UT

    1. Southern Utah Peering Regional Network (SUPRnet)

Salt Lake City, UT

    1. Salt Lake Internet Exchange (SLIX)

San Antonio, TX

    1. CyrusOne Internet Exchange (CyrusOne IX)

San Diego, CA

    1. San Diego NAP (SD-NAP)

San Francisco, CA

    1. San Francisco Internet Exchange (SFIX)
    2. San Francisco Metropolitan Internet Exchange (SFMIX)

San Jose, CA

    1. AMS-IX Bay Area (AMS-IX BA
    2. CoreSite – Any2 Northern California)
    3. Equinix San Jose / Bay Area Exchange (Equinix San Jose)
    4. NASA Ames Internet eXchange (AIX)
    5. MAE West

San Juan, Puerto Rico

    1. Internet Exchange of Puerto Rico (IX.PR)
    2. Puerto Rico Bridge Initiative (PRBI-IX)

Seattle, WA

    1. Megaport MegaIX Seattle (MegaIX Seattle)
    2. Pacific Wave Exchange in Los Angeles and Seattle (PacificWave)
    3. Seattle Internet Exchange (SIX)
    4. Seattle Internet Exchange (9000 MTU) (SIX Seattle (Jumbo))
    5. Seattle, WA Equinix Internet Exchange Seattle (Equinix Seattle)

Sterling, VA

    1. CyrusOne Internet Exchange (CyrusOne IX)

Tampa, FL

    1. Tampa Internet Exchange (TampaIX)
    2. Tampa Internet Exchange (TPAIX)

Tulsa, OK

    1. LiveAir Tulsa IX

Vienna, VA

  1. Equinix Internet Exchange Vienna, VA (Equinix Vienna (VA))
  2. MAE East