Post B41y8qShynOZ2MMMd6 by mwl@io.mwl.io
(DIR) More posts by mwl@io.mwl.io
(DIR) Post #B41y8orRvpzu4Vv5uq by mwl@io.mwl.io
0 likes, 0 repeats
@dch I have not, good idea! maybe virtio as well? Gonna play around some this weekend...
(DIR) Post #B41y8p3rBhvEgzj0Fs by Tubsta@soc.feditime.com
0 likes, 0 repeats
@mwl @dch I did as part of my testing. I use NVMe devices for everything, no matter the underlying storage (thanks @ctuffli ). FWIW ahci-hd should only used for compatibility requirements (usually to get virtio drivers onboard for Windows guests) as it dog slow.
(DIR) Post #B41y8pKWBlFXWfWJE0 by ctuffli@mastodon.online
0 likes, 0 repeats
@Tubsta @dch @mwl hopefully goes without saying, but yell at me if a virtual shelf of N NVMe disks in bhyve doesn’t work
(DIR) Post #B41y8pWDUGbi6wzeSW by Tubsta@soc.feditime.com
0 likes, 0 repeats
@ctuffli @dch @mwl You might know the current state, what is the pci device limit currently in bhyve? I think it is 32 but might be wrong
(DIR) Post #B41y8pf4xJhEYR8jGy by ctuffli@mastodon.online
0 likes, 0 repeats
@Tubsta @dch @mwl bhyve "slot" specification in the max is bus:pcislot:function where bus is 0-255, pcislot is 0-31, and function is 0-7. If the maths don't fail me, that is around 64k-ish devices
(DIR) Post #B41y8pty3xbdIc6cTo by dexter@bsd.network
0 likes, 0 repeats
@ctuffli @Tubsta @dch @mwl So, we talking bhyve or vm-bhyve?I would not be surprised if vm-bhyve doesn’t assume that, “no one would ever want more than 8 devices”, or at least devices of one type.Have you tried raw bhyve?
(DIR) Post #B41y8q4bQQ73pb573Y by mwl@io.mwl.io
0 likes, 0 repeats
@dexter @ctuffli @Tubsta @dch vm-bhyve, because I'm lazy
(DIR) Post #B41y8qFalYu4NgDtBY by Tubsta@soc.feditime.com
0 likes, 0 repeats
@mwl @dexter @ctuffli @dch No, not lazy. You just want consistency because fat fingers on a massive native command line just gives you headaches you don’t need (well that is my excuse) #repeatable #bhyve
(DIR) Post #B41y8qShynOZ2MMMd6 by mwl@io.mwl.io
0 likes, 0 repeats
@Tubsta @dch @dexter @ctuffli bhyve seems to work well, but yeah, those command lines are :flan_yikes:
(DIR) Post #B45TbVFdbFwlhiGrEe by dgeo@mamot.fr
0 likes, 0 repeats
@mwlI use 48+ disks zpools on bare-metal hardware, did not see any limit here...
(DIR) Post #B4JvioflJ8dctYClUW by mwl@io.mwl.io
0 likes, 0 repeats
Finally getting back to my 72-disk draid bhyve VM for #openzfsmastery. Turns out that setting ahci_device_limit="8" in the vm-bhyve config let it boot just fine.Thank you everyone for your suggestions, but especially @dch for this one!
(DIR) Post #B4Jwdo3gGQ19UpZFmC by mwl@io.mwl.io
0 likes, 0 repeats
@zrail @dch they're sparse zvols in one vm.It's not like I have the space for that kind of shelf, that many disks, or a sensible amount of data. But it will let me produce a well-caveated discussion of draid for the book.
(DIR) Post #B4K8KWITH7emMG3VxI by mwl@io.mwl.io
0 likes, 0 repeats
@rowens @zrail @dch see my earlier toot about my office lacking enough power to run the heater and printer simultaneously :flan_tongue:
(DIR) Post #B4K8KWWIRiiR38WYVM by wollman@mastodon.social
0 likes, 0 repeats
@mwl @rowens @zrail @dch So the 60-drive chassis that we just retired wouldn't be to useful to you even after scrounging ebay for the right caddies to plug drives in... (It's super frustrating that 60 is bigger than most of our users would like and smaller than most of our vendors want to sell -- and I can't fit 104-drive enclosures in our cabinets.)
(DIR) Post #B4K9sxprUfz1ZEf6au by mwl@io.mwl.io
0 likes, 0 repeats
@wollman @rowens @zrail @dch ha! Thanks, but no. I appreciate the thought, though.(also, the thought of getting that up the stairs to my office.. ugh, I'm old!)