[HN Gopher] Ask HN: How would you bootstrap a telco?
___________________________________________________________________
Ask HN: How would you bootstrap a telco?
I'm based in Vancouver, work remotely for a global company, and the
outage yesterday was a huge pain in the butt for myself and many
Canadians I work with... I am appalled that 911 services weren't
even working in some cases. The telcos here are terrible. I'd love
to hear how HN users would think of bootstrapping a telco? We've
had a few startups try and disrupt things in Canada (e.g., Wind
Mobile). They were funded by large global investment groups and
eventually simply got acquired by the incumbent telcos. The other
approach I've seen is piggybacking on existing telco
infrastructure. This is required by law since the Canadian
government subsidizes telco infrastructure... It's not uncommon for
the telcos to renege on their contractual/legal obligations. Plus,
this doesn't create any redundant networks. So: how else can one
launch a telco? What creative solutions have folks seen? I'm
thinking: VOIP-only service that has direct access to the Internet
backbone in Toronto, Vancouver, etc. so you can at least have
completely separate service in major cities... Or crowdfunding 5G
towers (though I imagine spectrum licensing is an issue here).
Would love to hear what other HN readers think.
Author : cl42
Score : 80 points
Date : 2022-07-09 14:55 UTC (8 hours ago)
| foobiekr wrote:
| There are plenty of wireless micro-ISPs in the Bay Area. Our
| small office used a gigabit wireless service to augment xfinity,
| my sibling has a wireless via Sail, there's Ridge Wireless, Wave,
| and so on.
|
| It would probably be worth talking to one of these to find out
| what the real experience is like. In my experience, people will
| gladly talk about what they've learned.
| gsatic wrote:
| Happened in India recently. Read the Reliance Jio story.
| kjellsbells wrote:
| Just wanna add one thing I've not seen below: telco is heavily
| regulated, so you definitely need to get a lawyer to either guide
| you thru the minefield, eg know exactly what phrases and service
| offerings stay on the less-regulated side of the law and which
| dont. Not all regs are bad (eg anyone can call 911, anywhere, and
| the network must figure out where they are) but its not for the
| faint hearted, and some might argue that the big telcos like it
| like that.
|
| Also, if you are going to do anything with voice, public cloud
| may not cut it. The reliability bar for voice services is
| insanely high. Sub 10ms failover between elements while
| preserving the voice media stream type of stuff. Five or six
| nines availability, or else the regulator beats the shi* out of
| you. This is not an outage model where your COO can put out a
| chatty tweet saying "sorry and heres a $10 credit."
| muttantt wrote:
| > Also, if you are going to do anything with voice, public
| cloud may not cut it.
|
| Non sense. I run a very successful over-the-top VOIP provider
| using commodity dedicated servers in multiple states, some even
| on Digital Ocean. Essentially no issues related to
| connectivity, be it business customers or residentials.
| elchief wrote:
| > Five or six nines availability
|
| Post is about telecom service that was down for a whole day
| iampims wrote:
| Take a look at https://www.helium.com/
|
| They have nascent 5G support, and it's all "owned by the people".
|
| Not a big fan of crypto shenanigans, but Helium comes as close as
| possible to crypto that kinda works and solves a problem.
| donohoe wrote:
| Can't do this in my building (NYC) due to line-of-sight issues,
| but worth investigating this:
|
| https://www.nycmesh.net/
|
| https://www.nycmesh.net/faq
| darepublic wrote:
| Armchair ISP bootstrapper comment: try to start in one location.
| And fuck permits, go full pirate on these assholes. Become
| popular for providing exceptional though very limited service in
| a small geographic area. Then social media campaigning to paint
| the incumbents black. Try to parlay from there.
| wahnfrieden wrote:
| Given the incumbents here, you might want to consider starting
| with dismantling capitalism over seeking market-driven solutions
| for cronies
| e-master wrote:
| Call me crazy, but I never had issues with Canadian telcos - I
| start to think I'm some statistical anomaly :). I used Rogers,
| Bell and Telus for cell - all worked just fine; other than their
| barely functioning websites I never had a problem. Are they
| fixing prices? Absolutely, it's an oligopoly, same as many other
| things in Canada; I think Rogers has a 40% profit margin, that's
| rich if you ask me. Regarding cable internet - there are some
| cheaper companies such as TekSavvy and CarryTel - they use the
| big guys' infra as a backbone, but cost 10-30% cheaper.
|
| My point is that up until this last event I was cognizant of the
| fact that the prices are too high, but was ok to pay for the
| quality I got, would have laughed in the face to anyone saying
| LTE coverage or speed are bad.
|
| As a side note about the outage - I think a lot of people
| underestimate incompetence in large companies. I often marvel at
| how for example large Canadian banks are making record profits
| with many disfuctioning teams in critical positions - a situation
| that got much worse with the wave of the great resignation. If I
| were to bet, I'd say some incompetent team broke the infra, so
| there's no need to look for conspiracy theories, people are just
| not competent in their positions and make mistakes when something
| has to be changed and they have no more excuses to avoid that
| change.
| yung234 wrote:
| Worked for a large corp for many years. Lots of incompetence.
| If middle mgmt were properly incentivized they could probably
| have reduced head count by half or more. But if all large
| companies did this there would be mass unemployment.
|
| Not surprised to see Rogers blow up. Looks good on them.
| Scoundreller wrote:
| I think there's an opportunity to leverage Starlink's concept at
| a municipal/regional level. Maybe Starlink will do this to sell
| into denser regions with miniaturized equipment. Get a high
| enough vantage point for the remote end, on the CPE side, issue
| steerable array antennas that can be self-installed.
|
| As demand increases, you could setup more remote ends and enough
| steerable array antennas can automatically re-aim and new subs
| will point to the new one.
| ISL wrote:
| I would look hard at the entire history of Tucows' efforts with
| Ting Mobile and Ting Internet. Lots of important lessons there.
| brailsafe wrote:
| Wind was interesting in that they got started, afaik, by
| operating on different bands entirely that didn't have the reach
| of any others, and then offered ostensibly better plans on that
| spectrum. It was kind of rough at the start though, and you were
| basically roaming outside of a few areas. Also, only a few phones
| at the time had the right radios.
| interestica wrote:
| Wind (Freedom) wasn't simply acquired. It was hampered and
| restricted by existing Canadian laws to do with foreign
| ownership.
| bcrl wrote:
| Basically, you start by getting giant piles of cash. You then
| start trying to build out new network assets. Then you run into
| the incumbents cheating on the permitting processes and utilizing
| market power to take or retain customers. I've been through this.
| And the municipalities don't even own up to their obligations
| under CRTC frameworks and delay MAAs for a year "just because".
| It's painful.
|
| Becoming the third player is an uphill battle in all regards.
| Fibre customers are _very_ sticky, and getting people to change
| is nearly impossible. Bell is particularly awful as a competitor
| as they steeply discount the first 1-2 years: want gigabit fibre
| with all the TV channels and phone service for $60 /month along
| with a $200 prepaid Visa gift card? Sure! Then month 13 comes
| along and the bill becomes $300 per month. As a competitor, there
| is absolutely no way I can offer a similar package or retain a
| customer when they take that package. Want that deal as an
| existing customer? No way!
|
| On top of all that, Canada is a relatively small market that is
| very spread out geographically. If you want to provide coverage
| similar to existing carriers, you have to build a lot of
| infrastructure. Nobody can afford to do that. Plus there are
| buckets of cash being rained down on other market players under a
| number of programs, so your competitor may be running on assets
| that cost 50-100% below your cost to build a similar
| infrastructure.
|
| The easiest thing for you to do is to get 2 internet connections
| from different carriers that do not share networks. In most
| markets the telco and cableco run different networks. My Telus
| and Bell services continued working fine yesterday. Heck, my 3
| different Rogers wavelength services we not impacted at all
| (although L1 services have other failure modes that take longer
| to recover from since there is no automatic rerouting as with L2
| networks).
|
| I'm pretty sure you really don't want to start down this path...
| kevin_nisbet wrote:
| > The easiest thing for you to do is to get 2 internet
| connections from different carriers that do not share networks.
|
| This is not as easy as it may seem, as it's common for the
| carriers to lease from each other in the backbones. Telus and
| Bell for cellular services share a radio network, each cell
| site connects to both the Bell and Telus core networks. The big
| Bell backbone outage in the Toronto area, 6 or 7 years ago, had
| impacts on all of Bell, Telus, and Rogers networks. As for
| cellular services we had glitches nationwide that had to be
| fixed during that backbone outage.
| icehawk wrote:
| And even if they don't share networks they might share
| _physical space_.
|
| I had an incident where two different trans-Atlantic links
| from two different providers, shared a 20 mile stretch of
| conduit.
|
| So, of course, that's exactly where the fiber cut happened.
| bcrl wrote:
| In the case of the Rogers outage, other carriers / ISPs were
| not offline. The Rogers problem appeared to be entirely in
| L2/L3 -- switches and routers were not configured correctly.
| The other telcos were not impacted as they run their own
| routing infrastructure. Basically, make sure your 2 ISPs have
| their own ASNs and transit providers.
| kevin_nisbet wrote:
| Sorry, I think you missed the point I was trying to share.
| The large providers all lease infrastructure from each
| other, so it's unsafe to assume that using two providers
| creates a total diversity in access. But you're correct, in
| that failures can happen at different layers, that may
| isolate outages to one provider or even a subset of
| customers.
|
| The outage I cited, only occurred in the Bell Wireline Core
| network, but caused service impacts in Bell, Telus, Rogers
| and other networks for Cellular, Internet, etc services.
| bcrl wrote:
| Again, it depends on what you're leasing from other
| carriers. Yes, if you're purchasing an L2 path from a
| carrier you're relying on their switches and routers to
| pass traffic correctly, but you bypass their routers and
| switches by purchasing an L1 wavelength circuit between 2
| points. That's one of the main reasons that wavelength
| circuits are used (in addition to the guaranteed
| capacity).
|
| Third party carriers using fibre local loops from another
| carrier would not have gone down like TPIA users did on
| Rogers' network yesterday. I strongly believe that access
| to fibre local loops is needed in Canada. Leased copper
| local loops enabled plenty of competition between small
| local ISPs in the 1990s and early 2000s while they could
| still deliver speeds that were sufficient for internet
| users of the time. Having a tariff for local fibre
| circuits similar to the old LDDS tariff would bring back
| that vibrant competition we once had. Bonus: it's a lot
| harder for incumbents to put together bogus cost studies
| as compared to the failed disaggregated regime the CRTC
| has wasted nearly a decade on now.
| icehawk wrote:
| _Heck, my 3 different Rogers wavelength services we not
| impacted at all (although L1 services have other failure modes
| that take longer to recover from since there is no automatic
| rerouting as with L2 networks)._
|
| Sort of, our wave circuits will automatically reroute but we
| had to shell out the $$ for protected waves and then explicitly
| mention they had to be configured to be auto-revertive as that
| was not the default behavior for that telecom provider. Then L2
| and L3 don't notice when it shifts between the working and
| protect paths, it happens fast enough the only visible change
| then is the latency difference between the two.
| bcrl wrote:
| In some regards it's better to get another wavelength circuit
| going to another city to pick up geographically diverse
| transit.
| icehawk wrote:
| True, our use case is a bit different as the waves go
| between PoPs/DCs/Offices so the only diversity can be in
| the path itself. Faster to have the path switch over and
| have L3 re-converge only when the working and protected
| path are unavailable.
| kevin_nisbet wrote:
| So this question really depends on what the objective is. This
| really seems to be a knee jerk reaction to a big outage, and an
| overall dissatisfaction with the big players.
|
| If the objective is to compete on a national scale, we're talking
| capital investments of something like $10 billion CDN per year.
| As I recall when I worked in the industry Bell was spending 5-6
| billion per year on capital investments, and if you're starting
| now you're playing catch up, so probably need to double that for
| a decade or two to catch up.
|
| And that's just capital investments, equipment, software, and
| cables in the ground. Can this be run way more efficiently, and
| with less overhead in billing systems and pet projects and the
| like, sure. But use these numbers just for the realm we need to
| be in. Let alone the people to build, maintain, upgrade, monitor,
| respond to outages, etc. Just to try and put that into
| perspective, the entire federal government spent about $440
| billion last year. So we're talking a good 3-4% of the federal
| budget, for what one of the big 3 is spending. And keeping in
| mind, they have a lot of power to compete on price and features
| wherever you get started.
|
| So I'm personally believe that difficult to duplicate
| infrastructure should be government run. It's simply impractical
| to go and say we'll have 5 or 10 or 15 competitors, and startups,
| and bankruptcies when the cables in the ground are so capital
| intensive. We don't run 15 power lines to each house so everyone
| can change on a whim. Why are we pretending we can run 15 fiber
| cables and do the same thing. We don't really want one new big
| competitor, we want a vibrant market of new approaches and
| failures to occur.
|
| So set that policy tirade aside, and let's scope down our
| expectations from nationwide.
|
| I get internet from a small provider, who mainly serves multi-
| dwelling buildings in the Toronto area. What they've been able to
| carve out, is a niche where it's cost effective to compete and
| start from. They're basically leveraging some, I think it was
| Industry Canada rules changes, that basically say that the telcos
| aren't able to claim ownership of the wiring inside a building.
| The provider I use, then just hired some Bell technicians, bought
| the same DSL remotes Bell was using for the same services,
| purchased a leased MPLS or similar line back to their data center
| to the basement of the building, and started making deals with
| Condo Boards and building owners to offer services. So an
| internet connection, DSL remote, and some technicians to
| terminate wires was what they needed to get started. Of course
| it's more complicated than this, but that's what it sort of
| boiled down to.
|
| The DSL remotes have a SIP client that can create a phone
| connection, so you can get home phone, and they built a cheapo
| IPTV service with some android boxes. Give away telephone, TV and
| internet for free in the common areas of the building for
| advertising.
|
| 911 services, they just contacted with whoever provides normal
| 911 services for voip providers.
|
| When I switched to them, it was cheaper than I got with an
| employee discount in the industry. Last year, they put fiber in,
| so I've got unlimited gigabit fiber for half the price or less of
| the big companies. The Big companies of course, now offer
| unpublished promotions from time to time that are only available
| to my building. This of course doesn't cover cellular service,
| but we're back to that capital intensive replication of
| infrastructure.
|
| I'm glad I'm out of this industry.
| stevenjgarner wrote:
| Having deployed extensive optical networks both aerially and
| underground throughout Iowa, my recommendation would be to locate
| a community that would value your contribution as a starting
| point. I would then deploy physical plant for specific customers
| overbuilding the fiber optics (e.g. if the customer wants 12
| fibers, deploy 48 selling 12 and keeping 36 for your own future
| use).
|
| In my experience VERY few people in the optical networking space
| understand how to engineer a municipal fiber optic network - they
| invest thousands of dollars per customer when it can be done with
| best practices for well under $1k per customer (let me just say
| that 802.1w RSTP is your friend). You need to combine all the
| different ISO networking layers into a SINGLE business model (ie.
| physical plant and Ethernet/VLAN circuits all by the SAME
| engineer not by different departments, otherwise things get
| unnecessarily over-engineered).
|
| Even more important than the technical engineering is the
| financial engineering. Once you understand you will never produce
| more capital from selling customers than it will cost to
| provision those customers, you need to consider more advanced
| financial engineering models - the one that worked great for the
| cellular industry and several optical networks is commercial
| paper. Find a lawyer that REALLY understands commercial paper -
| then every contract signed by a customer actually IS cash and
| does not need to be converted to cash. It's one of the only ways
| I know of releasing capital invested into physical plant.
|
| Good luck!
| foobiekr wrote:
| This is excellent advice.
|
| (Though I am curious why RSTP is so deserving of a call out - I
| mean, yes, but there's a lot of basics here that someone
| starting out would need to know, including things like avoiding
| fate sharing and basic planning, equipment realities, etc. if
| laying fiber.)
| stevenjgarner wrote:
| > curious why RSTP is so deserving of a call out
|
| I consider RSTP a core foundational technology for FTTH, not
| some condiment sprinkled on the network afterwards. Plus it
| is the Ethernet technology I find most typically missing from
| FTTH network design.
|
| Let me explain by way of an example: imagine a typical small
| town, population less than 50,000, with 2.60 people on
| average per household => less than 19,231 households.
| Traditional FTTH networks were modeled much like a POTS
| network in a star pattern with a dedicated optical fiber to
| each household from a central office. That then evolved (or
| should I say got bastardized) to a HFC (hybrid fiber coax)
| model by the MSO's as they already had a coax local loop
| going to each house. Clearly any kind of centralized
| distributed FTTH is doomed to financial ruin.
|
| As an alternative, image this: same community. You get the
| local school district as your first customer (or other
| geographically dispersed participant), where you deploy a
| ring of fiber circumnavigating the community, interconnecting
| every school. As above you over build their fiber, keeping
| some for yourself. Of course, you will have a peering point
| (or two) to interconnect 10 gigabit Ethernet with a tier one
| provider. You now have a ring of optical fibers less than a
| few miles from every home in town (typically less than a
| mile). Built on top of RSTP, the ring of fiber gives every
| node on the network route-diversified redundancy and
| protection. But not every residential customer is going to
| sign up with you, so the last thing you want to do is build
| out to every home. Instead you build out from your ring to
| those households that do sign up, where every home has a
| route-diversified optical connection extending the RSTP ring
| to the customer premise. This topology negates the need for a
| massive star network emanating from a central office.
| Literally with a handful of fibers you can provide for an
| entire community at a standard of engineering and quality of
| service the MSOs cannot even fathom. Massive reduction in
| physical plant requirement, massive reduction in capital per
| customer.
| derefr wrote:
| Can you dumb this down for someone who understands Internet
| networking (incl. internet backbone routing), and vaguely
| understands shared-medium networking like wi-fi and
| cellular networks, but who knows next to nothing about
| coax, and especially nothing about fiber?
|
| What does a fiber loop (bus?) let you do hardware-wise,
| that a star topology wouldn't? Either way, you're running a
| piece of fiber to the home from _somewhere_ , and
| connecting it to your network _somehow_ ; and I would
| assume that it's that connection that's inherently
| expensive, because of the equipment involved. In star
| topologies, that's a switch, or a hierarchy of switches.
| For a shared medium, that's a hub--or, I guess, some kind
| of direct splice, if that's somehow cheaper in TCO.
|
| Is the goal of using a loop, to avoid the cost of having
| neighbourhood-distribution-box optical network switches,
| instead directly splicing customers onto the ring, making
| the customer's fiber "part of" the shared medium of the
| loop, and then allowing customer equipment to directly push
| packets / transmission cells onto the loop?
|
| If so, why does it sound like you're implying that in a
| traditional FTTH star topology, there _wouldn 't_ be per-
| neighbourhood optical network switches, and instead only
| one wide-branching layer where all customers' lines
| terminate directly at one central location? That seems kind
| of ridiculous. Are optical switches really that expensive,
| that those are the only two options? Especially when
| neighbourhood-distribution-box optical switches would be
| relatively-low fan-in + relatively-low bandwidth, and
| therefore could be built to use simple low-frequency TDM
| circuit-switching protocols like coax's ATM?
|
| Also, is client-accessible fiber head-end equipment really
| trustworthy to do TDM (FDM?) onto a shared fiber medium,
| without ruining it for everyone else? We don't trust client
| equipment to do that on enterprise Ethernet networks (if we
| can at-all help it); we use switches instead of hubs, and
| we even put VLANs in place to further isolate flows in the
| merged path for QoS purposes. Is the difference just a
| matter of a fiber provider only certifying specific
| (presumably robust) equipment; and of that equipment not
| being coupled to power delivery issues of an arbitrary host
| device? Is that enough to get you peace-of-mind that your
| shared-medium network won't be being blasted with noise
| from new failing head-end equipment every few days?
| geocar wrote:
| > What does a fiber loop (bus?) let you do hardware-wise,
| that a star topology wouldn't?
|
| Each site has two links, left and right, that have
| engineered different paths.
|
| If your link is to home base, chances are your second
| link to home base is going to take the same path.
| ftth_finland wrote:
| The misunderstanding is due to the grandparent getting
| things wrong.
|
| It's the building out of fiber to each premise that is
| the expensive part, not the topology or the equipment.
| Those are rounding errors. What is expensive is
| undergrounding the fiber or hanging it on poles.
| stevenjgarner wrote:
| EMPHASIS > It's the building out of fiber to each premise
| that is the expensive part, not the topology or the
| equipment.
|
| That's it in one sentence. So you want a topology and
| engineering model that will allow you to reduce your
| total network cost.
| ftth_finland wrote:
| Spanning tree is an exceptionally bad protocol choice for a
| ring network. It's not like you want to create a giant
| layer 2 network anyway.
|
| I suggest you look into routed networks and redundant GPON
| topologies.
| bcrl wrote:
| On top of that: avoid engineering consulting firms. Hire an
| engineer that can stamp and sign off on plans to do the work
| for you. Pole surveying is not all that expensive to do
| yourself, and there are multiple technology solutions in that
| space now that didn't exist 10+ years ago. The first
| engineering consultants I had to pay were billing $6/m. Doing
| it myself with a P.Eng reviewing is under $1/m.
| stevenjgarner wrote:
| I cannot agree enough. Design engineering of an FTTH solution
| is not the domain of a single engineering discipline. If you
| "locate a community that would value your contribution as a
| starting point", that city council will move heaven and hell
| to ensure you have the rights to enhance their town. If you
| do everything underground, then the city can usually sign off
| on your easement rights and access (plus they will give you
| the friendly coordination of the water and sewer department
| and maybe other local utilities). Going aerially will be
| quicker and cheaper, but it will be fraught with more
| potential obstacles from the incumbents. Directional boring,
| plowing and trenching have become commodity priced
| construction costs ($ per ft), with massive competition. Many
| underground construction firms will give you the credibility
| you need to work responsibly and with accountability (bonding
| and liability insurance).
|
| Once the physical plant is design engineered, your Ethernet
| and Internet engineering is virtually unregulated. You will
| want specialists to set up your BGP constellation and other
| things that will only be done once, but if you have a good
| working knowledge of routing and Q-tagged VLANs and other
| core Ethernet technologies, you will be amazed how much you
| can do yourself.
| Scoundreller wrote:
| > I am appalled that 911 services weren't even working in some
| cases.
|
| This was the most surprising issue of the downtime. Rogers had
| _just_ enough layers lit up for 9-1-1 calls to be attempted (to
| only fail) on its completely dark network that cell phones didn
| 't/wouldn't failover that 9-1-1 call to the other mobile networks
| that were up.
| mikewarot wrote:
| You need new physical data routes, which isn't cheap. I'd start
| working on a local non-profit, low bandwidth effort first.
| Something that can be used in an emergency to route around
| damage.
|
| Also, see about using StarLink.
|
| Also, do you have ComCast in addition to the phone company? It's
| always good to have a backup route to the world.
| zinekeller wrote:
| > Also, do you have ComCast in addition to the phone company?
| It's always good to have a backup route to the world.
|
| While a reasonable comment in the US, in most cases the phone
| company and the cable company in Canada are usually one and the
| same or have a common parent (and if the Rogers-Shaw merger is
| closed would make this true to nearly 90% of Canadians).
| bcrl wrote:
| Yesterday's outage was not an L0/L1 problem, it was L2 and L3.
| Rogers wavelength services continued working perfectly fine (I
| have 3 x 10Gbps circuits from them). Even the Rogers owned data
| center my gear is in was fine as they were an acquisition that
| still has IP connectivity from other carriers (including Zayo).
|
| As the owner and operator of a small non-dominant carrier, what
| would really help is to have access to fibre local loops and
| transport at reasonable costs that reflect their 30+ year
| operational life. Existing fibre owned by the incumbents
| doesn't need to be duplicated. Unfortunately that's not the
| path the CRTC has taken to encourage competition. I really need
| to write a new Part 1 application on this.
| ttul wrote:
| Please do. And good luck with that. The level of national
| frustration with telecoms must have reached a new record
| yesterday.
| bcrl wrote:
| If I had infinite time, I would. It's taken a few years to
| recover from no fewer than 4 projects which were complete
| write offs in terms of profitability because my permits
| were delayed while the incumbent built out a neighbourhood
| for themselves. There's nothing better than a half million
| dollar hole in your financial plans thanks to delayed and
| denied permits.
|
| Starting to build a carrier is a really steep learning
| curve. You don't know what the rules are, and you don't
| know how and when to fight back against the incumbents.
| Hell, the CRTC is a huge problem in and of itself as it
| took 2 years to sort out a fairly simple filing by
| Videotron where they had permits improperly denied to
| access poles (see
| https://crtc.gc.ca/eng/archive/2022/2022-160.htm for the
| decision on that filing -- my comments were referenced
| several times in that decision).
|
| If someone really wants to start a small telco here in
| Canada, my number 1 recommendation is: do your own
| engineering. Engineering consulting companies will dig you
| a very expensive hole for projects that go sideways --
| avoid that. The second most important tip is to fight back
| and go to regulators the moment permits do not get issued
| in a timely manner. Learn from some of my pain to avoid
| your own expensive lessons learned.
| sneak wrote:
| The problem, as others pointed out, is regulatory capture. The
| market for telecommunications both in Canada and the US is
| entirely, completely corrupt, organized by unelected officials
| for the benefit of the few, at the expense of everyone in
| society.
|
| It's as good an example as any of the lack of real liberty that
| entrepreneurs have in certain industries.
|
| See also: banking and payments.
| h2odragon wrote:
| > doesn't create any redundant networks.
|
| I would say that redundant networks are _precisely whats
| required_.
|
| Selling a government on the idea is the hard part. Actually
| maintaining communications links, wires or fiber etc, is
| apparently so trivial a challenge as to be a minority of most
| phone companies actual business. They spend much more effort on
| billing and marketing.
| sgt101 wrote:
| Note. There is no internet backbone. You pay to peer your
| traffic.
| jcrawfordor wrote:
| A new startup ISP won't be peering, they'll be paying an
| incumbent provider for at least transit if not full access
| services (e.g. "dedicated internet access," basically
| commercial ISP service under contract terms that allow diverse
| uses including internet services). The term "backbone" is
| pretty useful here as the number of providers in North America
| with real global connectivity is fairly small; most ISPs with
| under thousands of customers are themselves customers of
| Centurylink/Lumen or Verizon, both of which have been steadily
| acquiring their competition in core internet connectivity
| (Level3 for example).
| bcrl wrote:
| You don't need to buy DIA from an incumbent. You can get L1 /
| L2 circuits from various carriers back to data centres where
| you can buy transit and peer with other ISPs using BGP. L1
| wavelength circuits did not suffer from the outage Rogers had
| yesterday, but the downside of wavelengths is that they
| require physical intervention to recover from failures.
|
| For exchanges, Ontario has the Torix exchange in Toronto for
| peering, and there's QIX in Montreal. Ottawa has Ottix, but
| it does not have many peers. There are others out west (as
| I'm in Ontario and don't pay attention to them as much).
| Scoundreller wrote:
| Torix has most of the peers you need for a majority of an
| ISP's traffic.
|
| Most consumer traffic is going to be to CDNs (Limelight,
| Akamai, Cloudflare), AWS, Apple, Google/Youtube, Microsoft,
| Netflix and the like if you're not big enough to have a
| private interconnect to them. It turns out the big internet
| companies like IX's too and don't want backbone providers
| being gatekeeper.
| perlgeek wrote:
| > You pay to peer your traffic.
|
| That's not the whole truth.
|
| At least here in Germany, there are publicly-ish peering
| points. You pay a small amount for colocating a single router
| in the data center, and each peering partner only accepts IP
| traffic for the IPs they feel responsible for (typically their
| own AS). Companies such as Google participate in those, so you
| can offload all of your Google and Youtube traffic at such a
| peering point. A small ISP can afford to participate in such a
| peering point, you only pay a fixed, monthly fee for
| colocation.
|
| However, you likely want to _also_ have a carrier to whom you
| can hand off all the other traffic that you don 't have direct
| peers for.
|
| (Source: I work for a company that hosts some of the POPs for
| https://www.peeringdb.com/ix/135)
| MattGaiser wrote:
| In Canada? You can't.
|
| Canada doesn't have a lack of competition. Canada has a lack of
| people willing to consider competition.
|
| https://www.youtube.com/watch?v=HlRPLBLkZK0
|
| Everyone I know whines about high telecom and internet prices. I
| know very few people willing to consider anything other than a
| lower price at Bell, Rogers, or Telus.
|
| Canadians are very risk averse people.
| agentultra wrote:
| While you're building the business try getting your municipality
| to subsidize access to poles and existing infrastructure.
|
| Join with other providers like Teksavvy to help the CRTC to break
| up the monopoly on the infrastructure.
| ksec wrote:
| Telecommunication industry is heavily regulated, both in terms of
| Spectrum and Services requirement. So practically speaking the
| only way to improve Teleco is either you have government support
| to open up the market for competition, i.e Politics, or you have
| another iPhone moment ( the odds of happening is infinitely close
| to zero ) where a vendor could dictate Teleco for Network
| improvement.
|
| I keep thinking the only way to break this anywhere in the world
| would be something like WiFi as a Mobile Network. Although this
| idea has been tried but never succeeded, practically because WiFi
| wasn't designed for such use in the first place. But recent
| improvement in WiFi 6 and WiFI 7 ( 802.11ax or 802.11be ) could
| solve this.
| pirate787 wrote:
| Starlink could also provide new backbone
| bcrl wrote:
| No. Starlink does not have enough capacity for that.
| ackbar03 wrote:
| Could you elaborate a bit? Not sure why the parent comment
| was voted down, I'm genuinely curious
| bcrl wrote:
| RF just simply does not have the capacity of fibre.
| Capacity upgrades in a satellite network take 5-10 years.
| Lighting up another wavelength on a glass fibre takes a
| few minutes to plug in new equipment at either end of the
| link.
|
| Right now I see usage across my network at around 4-5
| Mbps per subscriber at peak. It takes a few hundred users
| at those rates to overload a Starlink terminal with a
| gigabit port. Starlink is not a viable backhaul
| technology for anything but the smallest of networks.
| voisin wrote:
| For voice only?
| Nextgrid wrote:
| For remote areas I would think it could? If the choice is
| between Starling and absolutely nothing, the former can
| provide at least _some_ level of service. You 'd also
| expect such remote areas to not be so densely populated so
| a single Starlink backhaul can be enough.
| klabb3 wrote:
| As a curious bystander, why is wifi typically so short range?
| Today the frequencies of wifi and mobile networks seem to get
| increasingly similar, somewhere in single digit GHz. Yet my
| wifi router struggles with some drywall 3m away, despite
| several bulky antennas.
| bragr wrote:
| I believe consumer unregulated devices like wifi APs are
| limited to a broadcast power of 1W for one thing. Second, if
| you are indoors, it is more likely your phone is using a
| frequency band bellow 1Ghz
| zinekeller wrote:
| Different applications. Wi-Fi is designed to send bulk data
| but sacrifice coverage, while wireless telecommunications are
| designed to be reliable in thousands and have guaranteed
| coverage but sacrifices raw throughput.
| bcrl wrote:
| It's actually very easy to start a carrier in Canada from a
| regulatory perspective. The difficulty is in achieving scale
| and profitability while dealing with the incumbents trying to
| squash you like a bug.
| jcrawfordor wrote:
| "WiFi as mobile network" was sort of the concept behind WiMax,
| which was fairly aggressively trialed under a Clearwire/Comcast
| partnership in the US and didn't really go anywhere. Part of
| the disappearance of WiMax as as technology is simply that LTE
| won out as the 4G cellular standard rather than WiMax, but that
| came about in good part because WiMax was more suitable for
| fixed-point access than LTE and there was, and continues to be,
| little market interest in fixed-point wireless. Fundamentally
| it is extremely difficult for a wireless technology to compete
| with physical cable for fixed-point access, DOCSIS is pretty
| consistently keeping an order of magnitude ahead of the best
| wireless options.
| yellow_lead wrote:
| An old friend of mine setup an internet company in Canada. They
| used point to point wireless, and in some cases exchanged free
| Internet for being allowed to use people's high utility poles /
| structures, etc to beam wireless. This was for rural canada, so a
| bit specific though.
| moneywoes wrote:
| Start at one location in a time? Fellow Vancourite here as well
| the_only_law wrote:
| I'd make sure you have a damn good lawyer.
| j45 wrote:
| Bootstrapping likely means outsized leveraging of knowledge with
| smaller amounts of cash focused on sustainability from the start.
|
| Maybe there's a fintech type disruption on the horizon.
|
| This would be a fun project to hack on with 5-10 skilled folks
| with complementary skills to cover the industry, legal,
| financial, operational and the hardware and software angles to
| see what might be possible today. There have to be folks with a
| lot of experience in Canada that might be worth seeking out
| before beginning that feel exactly like you do and I would say I
| do too.
|
| How much cash investment would you start with?
|
| I'd be curious to how easy it can be for a non technical consumer
| today to access an mvno service that is turnkey and sells 3-4gb a
| month tablet type data plan (currently $10-20/mo) that can handle
| calls and sms via a carrier grade voip for $20-30/mo might be an
| interesting. It's not new, but it's not always easy to setup
| unless you're technical. Calls are secondary to data (WhatsApp
| calls) these days for many. Keeping it independent of one carrier
| doesn't preclude the fact that existing carriers have likely
| dealt with this type of approaches before. But it's ok to start
| and remain small and let growth be organic snarfed across
| multiple providers instead of one only.
|
| There may be something at the convergence or combination of some
| of the existing approaches or new tech that might be ready for
| the mainstream..
|
| - Building momentum by starting as a MVNO in a segment of the
| market where you can secure and build demand and grow services.
| There are some interesting data only services out there.
|
| - Perhaps a LoRa play. Great range. There seems to be some
| interesting gear that will pair Lora via Bluetooth to any phone.
| Software might be needed. Maybe there could be Lora beacons to
| assist with coverage.
|
| - What's possible with other open spectrums? Are there other
| unused spectrums that could be used in interesting ways, for exa
|
| - Begin with cheaper 3G and now 4g networks only like new
| providers.
|
| - Wifi calling overlap / voip coverage.
|
| - Could there be a way to setup a peer routing network between
| devices and to their internet connections?
|
| - Crowdsource/coop/non-profit focused on revenue and
| sustainability but not paying profit to shareholders. Non profit
| corporations do not mean mean people can't be paid extremely well
| for their work. Maybe there is a meaningful social
| entrepreneurship angle here. Might be some funding available too
| to provide access to those who are priced out of the market.
|
| - Maybe there is a way to use the enhanced networking
| capabilities of Linux only phones, or alternative setups.
| neilalexander wrote:
| > Perhaps a LoRa play
|
| Not even worth trying at all. LoRa link speeds will barely
| satisfy even the most frugal internet users, especially in a
| world where modern webpages can run into the megabytes.
| ThrowawayTestr wrote:
| Start burying fiber and erecting towers.
| Nextgrid wrote:
| When it comes to mobile, I have some experience there as I wanted
| to do the same and looked into it at a high level. The main
| takeaway is that it appears to be impossible and nobody who's in
| power to make it possible has incentives to make it so.
|
| The main problem with displacing incumbent telcos is that you
| need to run your own physical infrastructure - you can't really
| do so as an MVNO - no company will work against themselves and
| they will use various tactics (whether contractual, pricing-based
| or use technical limitations) to make sure you're not able to be
| "too" competitive with them.
|
| Therefore you need your own infrastructure - we're talking
| leasing land, building towers and purchasing expensive radio
| equipment every few miles, and you need to cover the whole
| country before you launch because nobody's gonna subscribe to
| phone service that only works in a small, specific area, and even
| though technically using roaming to make up for coverage gaps is
| possible, see the previous paragraph for why it won't happen (at
| least not sustainably).
|
| Then you've got the issue of spectrum. You can have all the land
| and towers and equipment in the world but in most countries you
| need to have a license to be able to broadcast on the regulated
| mobile spectrum. Those licenses are auctioned every so often, and
| the incumbent telcos are willing to throw their entire war chests
| at them because it's a guaranteed return over time.
|
| We're not talking millions or hundreds of millions here (the
| latter is just what you'd need for spectrum), we're talking
| _billions_. This is beyond what VC will fund, and isn 't a good
| match for VC anyway because it will take decades to recoup that
| investment, during which the regulatory or technological
| situation might change - a huge risk investors are unlikely to
| take without the guaranteed of _fast_ returns (which are
| difficult to do if your objective is to fix what 's wrong with
| the existing telcos rather than become just a different kind of
| rent-seeker).
|
| But let's say you've got the spectrum and land? Fine, let's talk
| about equipment. You've basically got a handful of vendors such
| as Ericsson or Huawei to pick from, and they're all terrible in
| their own ways. The equipment extremely expensive and may not
| even be sold, instead merely rented to you at a large price tag.
| It's often full of security holes and a pain to work with - see
| http://2014.hackitoergosum.org/slides/day1_Hacking-telco-equ...
| for an example. It's also a patent minefield so building your own
| is extremely difficult, though if you're in a location where
| software patents aren't recognized, you could potentially avoid
| the issue by using software-defined radio "breaching" the patents
| in software, but you'll still have to keep a good legal team on
| retainer and a big "rainy day fund" because the equipment
| manufacturers won't go down without a fight - and that doesn't
| mention the actual capital and talent needed to replicate decades
| of R&D.
|
| Finally, even if you've made it all the way through this and are
| now on a level to compete with an incumbent, they can out-compete
| you by selling the service at a loss, subsidized by nastiness
| elsewhere, whether unethical practices such as advertising or
| selling personal data, to outright fraud such as extra
| fees/charging for services not rendered/etc, knowing full well
| the law won't be enforced against them. This is why telcos are
| universally hated, but if the market prices have now aligned with
| the fact that it's subsidized by nasty behavior, there might be
| no way for an ethical player to compete.
| mawise wrote:
| This is a great read on setting up a WISP (wireless ISP)[1]. It
| might be US-centric, but has a lot of good resources for the
| process. Probably smaller-scale than what you're talking about
| but still a good read.
|
| [1]: https://startyourownisp.com/
| cletus wrote:
| Some random thoughts:
|
| 1. You didn't specify wireless or wired. It matters. With wired,
| your biggest problem is going to be getting permission to deploy
| towers and licensing part of the spectrum;
|
| 2. Wired avoids this but adds the problem (and the biggest cost)
| of the last mile. Where do you run those cables?
|
| a) Poles: you need permission for this and there can be a bunch
| of hyperlocal laws that will get in your way. For example, you
| may have to wait for existing telcos to move their wires up or
| down to make room and they'll take their sweet time.
|
| b) In trenches: you'll need a different kind of permission for
| this. Local conditions may make this more of less difficult (eg
| there can be alot of large rocks in the soil).
|
| 3. If you run a cable from a house and on a pole or in a trench
| where does it go? You need some kinf of substation for this
| depending on the max transmission length of the cables you
| employ. You need planning permission for that and teh big cost of
| the real estate required. If it's a hole in the ground, that's
| less of a problem. If it's a free standing structure containing
| switching equipment, expect residents to fight it on NIMBY
| grounds;
|
| 4. Depending on where you are you may face local or state laws
| that restrict your ability to do any of this. Some US states have
| passed laws against municipal broadband (at the behest of telco
| lobby). Telcos may have exclusive franchises with certain areas.
| This has been dismantled somewhat but you may find unofficial
| opposition anyway;
|
| 5. You're going to need installer for all of this and people to
| handle all the permitting;
|
| 6. Once it's built you need people to maintain it. People will
| come through and cut your wires or they'll be accidents like
| cheery pickers running through your cables or poles going down in
| a storm requiring cable repair.
|
| 7. Existing telcos are exceptionally good at playing this game
| and are typically in bed with local politicians who will make
| your life difficult.
|
| 8. Will you get enough customers? Just getting permission to
| build something will probably lead existing telcos to cut prices
| and lock in consumers with 12 or 24 month contracts to starve
| you.
|
| 9. Do you have a TV service? If not, you're going to lose a
| certain number of potential customers who still want this. If you
| do, the price you pay scales with your size. Verizon, Comcat or
| AT&T pay a lot less to content providers for their TV packages
| than you will. Telcos like to bundle TV and Internet so you may
| find your customers only pay $20/month extra to add Internet to
| theri TV package.
|
| 10. Who do you get upstream Internet connections from? It may
| well be one of your local competitors. Guess how that goes.
|
| This is a capital intensive business that doesn't reward
| overbuilds. The only way this can really work is with heavy
| cooperation from your local municpality. As mentioned there may
| be legal barries from them doing so (as noted, telcos don't want
| a repeat of Chattanooga).
|
| This'll work best in low population centers that are underserved
| or not served at all by existing players. Low density housing
| means getting trenching permission is going to be less of a
| problem. Potential customers will also be much more motivated if
| there other options are HughestNET or DSL.
|
| Starlink probably means even these customers aren't going to be
| as motivated as they would've been 5-10 years ago.
|
| In more rural areas you may even have the option of microwave
| relays that'll bypass a spectrum issues and it'll be easier to
| permission to be a directional wireless tower on someone's
| boundary.
| toomuchtodo wrote:
| Buy enough politicians to capture the ability to own and operate
| a monopoly telco run as a coop/utility. Put engineering talent in
| leadership roles, setting the necessary culture. Everything else
| is access to capital and effective execution of physical infra
| buildout and O&M ("the easy part").
| lormayna wrote:
| I have worked for a startup ISP almost 10 years ago. I was the
| first network engineer and I saw the ISP growing from scratch to
| more than 50k users and then collapse in few months for several
| terrible management choices. I don't know how is in Canada, but
| in Italy you need a lot of regulamentatory works before start to
| install the first equipment. Another point to consider if you
| want to scale up, is to have as much as processes as possible.
| Without workflows and procedures, the growing complexity will
| kill your ISP.
|
| > What creative solutions have folks seen?
|
| The idea that the ISP I worked for was simple: you are a very
| strong local reseller of a big ISP in a very limited area, but
| you don't have the flexibility in prices and services that you
| want. So "my ISP" will help you: we install a complete
| infrastructure in your area, you will pay us a fixed prices every
| month and we will manage the whole infrastructure on your behalf.
| As reseller, you have a portal to manage new orders and a support
| call center to solve the issues. It worked for several years.
| Feel free to drop me an email if you want more details, (my
| username) (at) gmail (dot) com
___________________________________________________________________
(page generated 2022-07-09 23:01 UTC)