[HN Gopher] "BGP at home": getting a DIA circuit installed at home
___________________________________________________________________
"BGP at home": getting a DIA circuit installed at home
Author : tripdout
Score : 200 points
Date : 2024-12-07 23:45 UTC (23 hours ago)
(HTM) web link (aaka.sh)
(TXT) w3m dump (aaka.sh)
| SemioticStandrd wrote:
| Hard to believe his company was okay with paying for all of this,
| but neat! I'm jealous.
| baby_souffle wrote:
| I know of only a few other people that have managed to get "real"
| fiber to the home.
|
| I am jealous of them save for the "your monthly ISP bill is
| similar to a car payment" aspect.
| therein wrote:
| So 300$ instead of $72.99 + tax and I get to not have Comcast?
| Sounds like a good deal.
| ocdtrekkie wrote:
| In most cases if you have Comcast, your best option for a DIA
| is Comcast Business. Which again, is drastically better than
| Comcast's residential service... but it's still Comcast.
| Adachi91 wrote:
| Comcast Business by itself is not DIA. You are still on the
| same CMTS as residential users. So you share the same
| resource pool. I have had Comcast Business for ~14 years
| and still fight with normal residential problems.
|
| Such as over-subscription and having to contact the BBB to
| finally get a non-"Let me look at my book, ah yes! it's
| your modem" response. I finally was contacted by the
| Technical Operations Manager to affirm "Yes [name] is
| correct, bandwidth demand exceeds the capacity of the
| system in their area. We are working on a permanent
| solution to allocate more bandwidth"
|
| That was 9 years ago, and I'm back again. I pay for the
| catchy "UP TO" 35 MBit/s upstream and can barely hold 2
| MBit/s during peak and about ~25 outside peak.
| theoreticalmal wrote:
| I'm just on residential Comcast and the slow upload
| speeds are such a pain in the butt. I understand most of
| the users in this area use the Internet to consume from,
| but some of us want to serve stuff too!
| therein wrote:
| My measured upload was 26.2Mbps. They offered me an
| upgraded upload speed package, I accepted it and am
| paying for it. It is still 26.2Mbps. I rebooted the CPE,
| waited months. Nothing changed.
| 15155 wrote:
| Why not Starlink?
| NBJack wrote:
| Starlink shared bandwidth amidst their network makes
| Comcast look generous.
|
| Between fighting for bandwidth amidst everyone else going
| to the same base station, random assigned IP addresses
| that occasionally end up with accusations of pirating you
| had no partaking in, and storms messing with your signal
| quality, I would heavily advocate against any reliance on
| it for business related operations.
|
| Starlink also does not offer static IP addresses.
| boredatoms wrote:
| TCP can get confused by satellite handovers
|
| https://blog.apnic.net/2024/05/17/a-transport-protocols-
| view...
| int0x29 wrote:
| Starlink is competitive with rural lines which tend to be
| DSL over unmaintained copper. If you have cable you will
| probably do better with that. If you are in a dense ish
| area just about anything will do better.
| ocdtrekkie wrote:
| Not all Comcast Business, no. There are different tiers
| of product for sure. I have "normal" Comcast Business in
| my house, where I have basic Comcast coax service for a
| little more money with some better service guarantees and
| a little less nonsense. (The biggest upgrade is being
| able to email an account manager. The biggest downgrade
| is no bundling with TV because Business won't install it
| in a house, so if someone in your house wants Xfinity TV
| you get two unbundled bills.)
|
| At work we deal with the sort of folks in this blog,
| where adding a link between a couple sites requires four
| or five months, multiple teams boring new fiber runs,
| etc.
| nvarsj wrote:
| It's more like $1000/mo I think. 300 would be a no brainer!
|
| In London UK we can get dedicated 1Gbps for ~450GBP/mo with
| 3k install fees.
| anotheracc88 wrote:
| Sounds good but what are peoples use cases. Versus getting a
| regional EC2 and working from that?
|
| Anything privacy related, running a TOR node etc. I get it.
| jonatron wrote:
| In this case, he said it's a fun and interesting way to add
| resiliency to the existing co-location they have.
|
| I frequently see people default to AWS, without any
| consideration of any other options. If you're running beyond
| a couple of small EC2 instances, it's worth looking at other
| options such as colocation. 37signals wrote about their cloud
| exit and how much they saved.
| ganoushoreilly wrote:
| I have a few machines I use to mirror / duplicate data from
| my tenants and client tenants when working on larger
| projects. It makes it much much easier.
|
| While Egress pricing is a pain in the ass on AWS, that's
| usually a small fee on the customer side comparatively.
| wutwutwat wrote:
| Good luck when the fbi inevitably kicks in your door after
| running a tor node in the US on a datacenter connection in
| your name, with the physical hardware sitting in your home.
| anotheracc88 wrote:
| Non exit node was implied
| ganoushoreilly wrote:
| I had "real fiber" run to our farm. 18 miles of cable was run
| and it took a year going back and forth. Originally it was a
| priced at $4500, which was to be a business expense. After
| install I had consistent issues with performance, after doing a
| lot of work to show them the issues and threatening disconnect,
| they upgraded the circuits for proper speeds and knocked it
| down to $2100. It's definitely still a car payment but it's
| much cheaper than it was.
|
| Ultimately I get the SLA, Direct access to cloud providers
| maximizing performance, i'm also able to host a few IP blocks
| which allow a couple internet facing machines.
|
| The home we sold recently had it pretty good too though, ended
| up with 3 5gig AT&T lines (no redundancy obviously) for only
| $450 a month total. That was pretty darn rad, even if the SLA
| wasn't the same.
|
| Benefit of working from the farm is that I can also snag some
| bw for personal use ;D
| dboreham wrote:
| In my experience it's just a case of knowing who to call and
| what to ask for. We are served by Spectrum. They will drop
| fiber into any building in their HFC service areas. We had it
| into an apartment building for several years then moved to
| another building with only coax, but they quoted me $2000
| install for fiber. Problem for me is that the fiber service is
| symmetric and so to get the kind of download needed for Netflix
| you have to pay for 200M+ upload. That's quite expensive for me
| so I passed. They don't offer 95% billing.
| cyberax wrote:
| Hey! That's my setup as well! I have one DIA connection and a
| backup VPN over a shitty Comcast business connection that gets
| terminated in a nearby datacenter.
|
| Getting an ISP to even _talk_ to me required quite a bit of
| sleuthing. And I was saying from the outset that I was ready to
| fully pay for the fiber run.
|
| Apparently, ISPs in my locality actually divide the city into the
| service areas. How the heck this is legal, I don't understand.
|
| Some tidbits from me: my ISP installed a big honking ADVA optical
| line terminal on my premises. Getting them to move it to their
| side and just provide me with an SFP connection is still my work-
| in-progress.
|
| The support is also outsourced into India, and getting them to
| understand what you want over the phone is... painful.
| Fortunately, the web ticket system is good enough.
| fallous wrote:
| Regarding the city divided into service areas, if you are in
| the US many cities provided franchise rights to cable providers
| that gave them exclusive monopoly rights to provide services in
| exchange for the cable provider spending all the money to
| install the lines and infrastructure. Most of those franchise
| deals were done in the 1970s or early 1980s, essentially
| mimicking the agreements that were in place for AT&T (or
| RBOCs).
| toast0 wrote:
| Franchise agreements have been non-exclusive by US federal
| law since a long time.
|
| Lack of competition is more about the cost to establish
| service in an area, and the ROI on service in an area with
| competition. It costs a lot to pull wires past a lot of
| potential customers and if many of them won't sign up because
| they already have a good enough option, it doesn't make much
| sense to do it.
|
| Cable and Telco compete because when cable was built, it was
| a completely different service, but they've both evolved to
| fill the same role.
|
| This is why mandatory line sharing is important for
| competition, and it's in the telecom act of 1996. But the FCC
| first said it only applied to telecoms, and then said it
| doesn't apply in remote terminals because of lack of space,
| and then courts said it doesn't apply at all because telecom
| and not cable isn't fair.
| fweimer wrote:
| In my very limited experience, this is common even for real
| business customers. I think it's because going from commercial
| ISP service to BGP service is not really an upsell. It's a
| completely different product category (carrier interconnect)
| and usually results in less revenue for the ISP (greatly
| reduced bandwidth charges etc.). As a result, sales folks
| aren't trained on it, and it is difficult to get through
| regular channels.
|
| If you get it, it can be great. Imagine your ISP calling you
| when you reboot your router.
| icedchai wrote:
| I'm running BGP to the home, over wireguard, from a couple of
| VPSes. It wasn't worth it to upgrade to DIA for me. It's
| mostly for hobbyist purposes so reliability is not really a
| huge factor.
| immibis wrote:
| Isn't it very typical for an ISP to run their line onto your
| premises and put the demarcation device on your premises with
| your power supply etc? Usually there _isn 't_ a "their side"
| where it still works. It's your premises, then up to kilometers
| of wiring, then the central exchange. Equipment required to
| terminate the wiring at your end obvious has to be at your end,
| not theirs.
|
| It's possible that the termination equipment could be a bare
| SFP and not a big box, but the ISP wants to be able to monitor
| the status of your connection up to the termination equipment,
| because that is their responsibility to keep online, and they
| can't do that if it's just an off-the-shelf SFP. They probably
| wouldn't agree to do it and still have any SLA.
|
| If there's a physical problem with the box (too big/loud) you
| can try negotiate for a different box but if you just want
| control over your network, sorry but that just isn't how it
| works. Your network starts at the demarcation, and you don't
| want to be responsible for speaking whatever protocols the ISP
| is using internally, either. Up to the demarcation, it's ISP
| internal network, and past that, it's your network, with a
| standard handover interface like Ethernet.
| kbolino wrote:
| This depends on the technology. When I had a PON connection
| (FiOS), the ONT belonged to the ISP but was on my premises.
| However, with the cable Internet I have now (Xfinity aka
| Comcast), there is nothing between me and the street save a
| splitter and PoE filter, both of which are passive.
| immibis wrote:
| But you have a cable modem. You either have one provided
| directly by Comcast or you have one that adheres to their
| standards.
|
| It used to be that you had to get a cable or DSL modem from
| your ISP. Now, cable and DSL networks are standardized
| enough that there's a good chance you can get a third-party
| one to work. It will be the same with PON in time.
|
| I think it's fine to be required to use your ISP modem,
| since different ISPs have different physical layer
| networks. It starts to suck when they try to force you to
| use their _router_. I think in the EU it 's actually a
| legal requirement that if they do, it has to support bridge
| mode (a.k.a. behave as only a modem).
| jhugo wrote:
| > Now, cable and DSL networks are standardized enough
| that there's a good chance you can get a third-party one
| to work. It will be the same with PON in time.
|
| It's mostly the same with PON now. I've always had
| success with the FS.com PON SFPs, once I get the
| necessary information for the connection to program them
| with (the difficulty of which can vary from "ask the tech
| installing the connection nicely" to "take apart the ISP-
| provided CPE and solder wires to the debug console
| pads").
| kbolino wrote:
| Comcast has supported standard DOCSIS modems for a long
| time now. I have some old ones lying around, but they're
| too old for modern speeds.
| cyberax wrote:
| > Isn't it very typical for an ISP to run their line onto
| your premises and put the demarcation device on your premises
| with your power supply etc? Usually there isn't a "their
| side" where it still works.
|
| There is, in my case. Their point of presence is about 800
| meters away, and they have a simple switch that aggregates
| connections. Their optical terminal on my terminal (a switch-
| sized 1-U rack-mounted box with ANNOYING fans) does all the
| traffic shaping, authentication, etc.
|
| > It's possible that the termination equipment could be a
| bare SFP and not a big box, but the ISP wants to be able to
| monitor the status of your connection up to the termination
| equipment, because that is their responsibility to keep
| online, and they can't do that if it's just an off-the-shelf
| SFP. They probably wouldn't agree to do it and still have any
| SLA.
|
| Sure, and having equipment on my premises makes it much
| easier to debug the issue. But they actually monitor the BGP
| session state, not the optical path.
| simonjgreen wrote:
| The ISP I own will give you dedicated if you desire. We charge an
| uplift for the installation (due to additional splicing) and a
| little bit more on the monthly price for the consumption of
| dedicated ports and cores rather than using the PON. However, as
| is the case with most provided services like this, the majority
| of the cost is covering the risk of the SLA. The likelihood of an
| outage is not too dissimilar to the likelihood of outage on PON,
| so really it becomes a financial and service guarantee more than
| an uptime guarantee. As a healthy middle ground, we will also
| offer BGP on regular services and we do a bit of a referral
| system with a couple of other friendly ISPs, who will also do
| BGP. I actually prefer multi homing to "dedicated" from a
| resilience perspective because it separates you from the entire
| network stack all the way to the transit and peering.
| NetOpWibby wrote:
| What's your ISP?
| lbotos wrote:
| click their profile https://1310.io/
| NetOpWibby wrote:
| Thank you! I clicked their profile on my phone while in bed
| and must've missed it.
| implements wrote:
| (Potential non-commercial customer) Checked the help page -
| couldn't see anything about static IP addresses, IPv6 etc?
|
| Edit: Ah, "Our network. You acknowledge that We may change your
| Internet Protocol (IP) address from time to time without giving
| notice unless You have purchased a fixed IP address from us;"
| anotheracc88 wrote:
| That was written by an investor. Weirdly... would make me more
| interested in them if I were someone who needed investment.
| marcus0x62 wrote:
| > While the SLA says 100%, don't expect perfection
|
| When you have an SLA, understand what it is: a financial
| arrangement whereby you can request a prorated refund for certain
| types of outages. It is not in any way a guarantee on the part of
| a provider that you'll experience even average uptime equaling or
| exceeding the SLA, just that they can pay out the fraction of
| customer requests for service credits they receive for the
| covered outages they have and still make money.
|
| The reality for the type of service the author of this post
| purchased is that for any physical damage to the fiber plant, he
| will experience _hours_ of outage while a splice crew locates and
| repairs the damage. Verizon might offer a 100% SLA, but they didn
| 't engineer it to even five nines of availability. That would
| require redundant equipment and service entrances at his premises
| along with path diversity end-to-end.
| Arcanum-XIII wrote:
| It's still a very high motivator to keep the service up though.
| It's not a guarantee of anything as you said but I've been on
| call for this kind of contract. And then you get very good at
| pointing fingers too. Not too sure about this though :D
| mikeocool wrote:
| Verizon must make some killer margins on a connection once it's
| up and running, given they're willing to eat the cost of 4
| employees and a police detail splicing fiber in manholes for a
| week.
| 293984j29384 wrote:
| Back in 2015, Time Warner Cable testified they make a 97
| Percent Profit Margin on High-Speed Internet Service for
| residential customers. I would guess Verizon is doing okay.
| betaby wrote:
| That was because according the accounting infra is mostly
| amortized for 'triple play' and TV was ( is ? ) highly
| profitable. Basically 97% math assumed that infra was built
| for TV channel delivery and the Internet is 'free bonus'.
|
| I don't work for TWC and have no their services. However
| friend of mine worked in past for a regional competitor of
| TWC around 2010 and explained the logic above.
| 293984j29384 wrote:
| Verizon made a net profit of 12 billion USB last year. The
| year before they made 24 billion in net profit. This is a
| highly profitable business.
| betaby wrote:
| Those billions are mostly from mobile. My point is
| specifically about home Internet.
| mastax wrote:
| Very similar to the process I took getting DIA to a commercial
| building. They said they spent $60,000 pulling fiber half a mile
| through a business park (we didn't pay this). We only have one
| ISP device in our rack (Verizon truly ships their org chart). We
| paid our contractor to have fiber installed from the DEMARC to
| the server room but apparently the ISP would've done that for
| free, oops.
|
| And yeah the quality of customer service we've gotten from three
| different business providers has been exceptional. It's crazy to
| have actual engineers you can call who know what's going on. You
| get what you pay for.
| pickle-wizard wrote:
| This was about 15 years ago, back when I was working at
| $MEGACORP we had an OC-48 running to our lab. We were having
| some problems with it. Thanks to 20 years of near constant
| layoffs all details of who was responsible for that circuit on
| our side were lost to the sands of time.
|
| I went down to the basement and saw a faded UUNET sticker on
| the demarc, but there was no circuit id on it. Some googling
| showed that through the years of corporate takeovers they were
| now owned by Verizon. So I called Verzion Business and
| explained the situation. The lady spent a hour on the phone
| with me, but we tracked down the circuit. The address listed on
| the circuit was a manhole up the street from our building. They
| dispatched a technician and we were back up and running in
| about an hour. They also put new circuit ID label on the demarc
| so we wouldn't go through that again.
| bumbledraven wrote:
| I'm astonished that Verizon didn't charge separately for
| installation. Seems like it would take many years months of
| service to recoup the cost.
| rssoconnor wrote:
| > ISPs tend to oversubscribe these services as well (where you
| and your 10 neighbors might all be able to sign up for 1Gbps
| symmetric service, but not everyone can leverage that full 1Gbps
| at the same time).
|
| How does this work for FTTH? I know nothing about fibre optic
| networks. I had the impression that each subscriber has their own
| wavelength, or rather a range of wavelengths that captures their
| bandwidth, and that does not overlap with other subscribers.
|
| Otherwise I have no idea how passive optical networks could even
| work.
| cesarb wrote:
| > How does this work for FTTH? I know nothing about fibre optic
| networks. I had the impression that each subscriber has their
| own wavelength, or rather a range of wavelengths that captures
| their bandwidth, and that does not overlap with other
| subscribers.
|
| The keyword to search for is GPON. It's multiplexed, each
| subscriber receives a few time slots in a shared wavelength.
| The transmissions from the subscribers don't collide because
| their time slots don't overlap.
| chaz6 wrote:
| In a typical passive optical network, one PON port is connected
| to 128 clients through the use of PLC splitters - unlike a WDM
| splitter which will insert or remove a specific wavelength,
| these simply split the whole signal. Where I work, that is a
| 4-4-8 configuration using 3 layers of splitters.
|
| The OLT (optical line terminal, head-end) will tell each
| ONU/ONT (optical network unit/terminal) how much airtime they
| can use to transmit - each ONT will take their turn in
| transmitting so as not to interrupt others. Part of this
| calculation is the distance the ONT is from the OLT - each ONT
| will be a different distance depending on the geographic
| location, which means each ONT will have a different latency.
| The ONU can request additional airtime if it has a large amound
| of data to transmit. The amount of airtime the OLT will
| allocate depends upon the CIR (committed information rate i.e.
| what will the ISP guarantee at a minimum) and the PIR (peak
| information rate - the maximum rate based on the subscribers
| service).
| rssoconnor wrote:
| TDMA. Now that is a name I haven't heard in a long time.
___________________________________________________________________
(page generated 2024-12-08 23:01 UTC)