Post ASoZew1IEa4uYLjO40 by selea@social.linux.pizza
(DIR) More posts by selea@social.linux.pizza
(DIR) Post #ASoZ5qAG0qJ8G1Bovo by selea@social.linux.pizza
2023-02-18T20:10:17Z
0 likes, 0 repeats
I tried out #longhorn on #kubernetes tonight, and to me it nice and easy, but it still make me uneasy for some reason.Most likely because I want to understand how it works under the hood.Time do dig deeper I guess.
(DIR) Post #ASoZD0p2mkkZL5gf9E by selea@social.linux.pizza
2023-02-18T20:11:32Z
0 likes, 1 repeats
I have never been a fan of "just do this and this magic thing will happen".
(DIR) Post #ASoZESCsSLhGYmWS9Y by meraord@mastodonsweden.se
2023-02-18T20:11:50Z
0 likes, 0 repeats
@selea för en sekund trodde jag du menade ko-sorten longhorns...!
(DIR) Post #ASoZIKgyKyNEPcRsNk by selea@social.linux.pizza
2023-02-18T20:12:28Z
0 likes, 0 repeats
@meraord Det är nummer 3, på andra plats är Windows Longhorn ;)
(DIR) Post #ASoZRRx8iA1FXNMGWW by sa0bse@chaos.social
2023-02-18T20:14:11Z
0 likes, 0 repeats
@selea Longhorn is great! I have some experience with Longhorn and it kinda "feels" magic to begin with.What it mostly does is storing filesystem images in `/var/lib/longhorn` and automatically replicate them around to have enough replicas.
(DIR) Post #ASoZew1IEa4uYLjO40 by selea@social.linux.pizza
2023-02-18T20:16:35Z
0 likes, 0 repeats
@sa0bse yeah, I have failed to understand how open-iscsi comes into the mix in my case - there is no block devices mounted on the hosts from what I can se :(
(DIR) Post #ASoZuD3VB7XHjR31U0 by sa0bse@chaos.social
2023-02-18T20:19:21Z
0 likes, 0 repeats
@selea I use it in production for a not very small website here in Sweden... however that doesn't mean that I'm exactly aware of exactly how the mounts are performed other than "it works", and "the backups and snapshots seems to work as well".
(DIR) Post #ASoZvJJ7Q4QrPYsSGW by joacim@mastodon.melin.org
2023-02-18T20:19:22Z
0 likes, 0 repeats
@selea
(DIR) Post #ASoa3XBHinCOkx8OrQ by selea@social.linux.pizza
2023-02-18T20:21:04Z
0 likes, 0 repeats
@sa0bse I dont doubt the ability of longhorn, I just doubt myself to be able to debug it if stuff goes south :P
(DIR) Post #ASoa5hdboXTZlbPuc4 by cardes@metalhead.club
2023-02-18T20:21:22Z
0 likes, 0 repeats
@selea I‘m interested to see what you find out. I‘m to anxious using storage provided on top of kubernetes because i ended up with broken storage every time i tried that. I really like the dashboard and configuration overview though.
(DIR) Post #ASoa64BNzdZBgjOCJc by selea@social.linux.pizza
2023-02-18T20:21:27Z
0 likes, 0 repeats
@joacim I need to
(DIR) Post #ASoapCkYzEEBI5Mnbc by selea@social.linux.pizza
2023-02-18T20:29:40Z
0 likes, 0 repeats
@cardes I am not really confident in running persistent storage entirely in k8s - yet.I feel that I lost control somehow, but I'll learn to live with that
(DIR) Post #ASob55ahN6bfijKKTw by selea@social.linux.pizza
2023-02-18T20:32:32Z
0 likes, 0 repeats
@cardes You know what, it basically feel like those appliances you buy from some vendors, where the instructions basically says - "put this in your rack, put it power and network - done"
(DIR) Post #ASohyUo3vDXRZqNfnM by cardes@metalhead.club
2023-02-18T21:49:48Z
0 likes, 0 repeats
@selea Yeah i can't have that. If i don't understand something i have to dismantle it till i know whats going on. After doing that with longhorn, openebs and rook/ceph i decided that my gut feeling is bad enough to not try that out in production 😅 K8s always feels a bit like letting go of control, thats just how things have to work because of the grade of automation. On the other hand that shouldn't gloss over bad design/architecture. 1/2
(DIR) Post #ASoi4BpLOpJIS2hBIm by cardes@metalhead.club
2023-02-18T21:50:50Z
0 likes, 0 repeats
@selea I once used an image registry on the same cluster as the storage (provided by the cluster) - that ended not so well 😂
(DIR) Post #ASokBTnLC6Ly72MWIK by killua99@social.linux.pizza
2023-02-18T22:14:31Z
0 likes, 0 repeats
@selea👀 (LURKING) #longhorn #kubernetes
(DIR) Post #ASolW2t0XDVSgzNpK4 by Oleksii@social.linux.pizza
2023-02-18T22:29:16Z
0 likes, 0 repeats
@selea @cardes I participate in a project where we are trying to run a lot in k8s. Running DB in your own K8s cluster is a difficult task.@selea please share your concerns.
(DIR) Post #ASoo0O86JgJPAp2UrI by linus@telegrafverket.cc
2023-02-18T22:57:23Z
0 likes, 0 repeats
@selea kubernetes makes me uneasy
(DIR) Post #ASpStZKCOJBlzKZ5PM by stemid@hachyderm.io
2023-02-19T06:35:34Z
0 likes, 0 repeats
@selea only started hearing about longhorn lately.I've been using rook-ceph for a few years now at work, and briefly in homelab at home before I dismantled that.As long as you have at least 3 nodes rook-ceph works just fine. and it's just as magic.but the number of nodes is important with rook-ceph. you should really have more than 3 for redundancy.
(DIR) Post #ASpgSmKBLhlPqUg0oK by anders@mastodon.cyborch.com
2023-02-19T09:07:33Z
0 likes, 0 repeats
@selea I actually ended up just setting up a nfs server and using that in StatefulSet configuration. It’s relatively simple and I understand everything that’s happening.
(DIR) Post #ASpnVpm5g3eR5t8Pom by selea@social.linux.pizza
2023-02-19T10:26:35Z
0 likes, 0 repeats
@anders Yeah that is what I am doing too,
(DIR) Post #ASpndv3nuQVPnuH8U4 by selea@social.linux.pizza
2023-02-19T10:28:02Z
0 likes, 0 repeats
@stemid I have thougth about trying out rook aswell, but what scares me is that it is using ceph - and ceph is not that fun to use on few nodes :(I have a ceph cluster toghether with some friends, that is close to 200TB, and I use the RadosGW for some stuff there
(DIR) Post #ASpngCQMK1KMEgudiS by valerauko@misskey.io
2023-02-19T04:52:24.114Z
0 likes, 0 repeats
@selea@social.linux.pizza be careful using it for high pressure workloads (like prometheus or similar frequent write databases) because it can eat a lot of resources syncing/replicating
(DIR) Post #ASpngCvuQiZ3oXfptQ by selea@social.linux.pizza
2023-02-19T10:28:26Z
0 likes, 0 repeats
@valerauko I was thinking about moving a psql db into it, maybe a calckey database, but I am not sure
(DIR) Post #ASpnnLCs4ElFV90chE by cardes@metalhead.club
2023-02-18T22:34:00Z
0 likes, 0 repeats
@Oleksii @selea What kind of DB are you talking about? I'm not really in the application side of things and more on the cluster administration side and we are using separate network interfaces and network cards on the nodes to handle storage traffic to external storage appliances like netapp ontap. Storage provisioning is then handled over k8s csi in assistance with an operator provided by netapp (Astra Trident)
(DIR) Post #ASpnnLqDhwEjTBQ31s by selea@social.linux.pizza
2023-02-19T10:29:44Z
0 likes, 0 repeats
@cardes Using external storage like Netapp CSI or Nutanix CSI (I have used that quite much) is something that I feel more comfortable with rather than running everything inside k8s.@Oleksii
(DIR) Post #ASq2Z3ur3U3r8MEPSa by stemid@hachyderm.io
2023-02-19T13:15:10Z
0 likes, 0 repeats
@selea to me it's been surprisingly robust actually. The problems that did surprise us could have all been caught by simply monitoring osd health status more closely. And so far in three years, three clusters and running it on virtual disks we've had zero data loss peppar peppar
(DIR) Post #ASqAJHThfBX4CCW8o4 by abdulocracy@toot.abdulocra.cy
2023-02-19T14:41:54Z
0 likes, 0 repeats
@selea abstraction?
(DIR) Post #ASqcXtIsciFsHQD4mu by selea@social.linux.pizza
2023-02-19T19:58:17Z
0 likes, 0 repeats
@sheogorath Yeah, but how it is capable of talking "directly" to the host and create those volumes?@sa0bse
(DIR) Post #ASqhKA344L8tvj5JtQ by sa0bse@chaos.social
2023-02-19T20:51:56Z
0 likes, 0 repeats
@selea @sheogorath I guess that's up to the deamonsets running in each node
(DIR) Post #ASqiFnUa5fAQlkRFL6 by selea@social.linux.pizza
2023-02-19T21:02:18Z
0 likes, 0 repeats
@sa0bseI still have lots to understand @sheogorath