Help with setting ipw reddit / freebsd ---------------------------------------------------------------------- Feelsnicy · 2022-06-14 08:15 I have FreeBSD 13.1 running on raspbery pi which i'd like to set as router for lan clients. It has 2 network cards, one for internet connection and one for lan clients. Except routing i also need to run vpn connection on freebsd and pass lan clients through it(clients shouldn't know they connected through vpn). Setup: ue0 - external wan iface, dhcpd ue1 - internal lan iface, static tun0 - openvpn iface ​ rc.conf hostname="freebsdPI" ifconfig_ue0="DHCP" ifconfig_ue1="inet 172.16.1.1 netmask 255.255.255.0" # gateway_enable="YES" firewall_enable="YES" firewall_nat_enable="YES" firewall_logging="YES" natd_enable="YES" natd_interface="ue0" ## dnsmasq_enable="YES" sshd_enable="YES" ipfw !/bin/sh wan="ue0" lan="ue1" ipfw="/sbin/ipfw -q" #Reset all rules: ${ipfw} -f flush ${ipfw} -f pipe flush ${ipfw} -f queue flush ${ipfw} add divert natd ip from any to any via ue0 ${ipfw} add allow ip from any to any in such setup lan clients get internet, but i dont know how to forward traffic through vpn connection e.g. tun0. I found working setup for Linux box and tested it and it works as expected, i need the same for freebsd. ​ #!/bin/bash IPTABLES="/sbin/iptables" # $IPTABLES -t nat -A POSTROUTING -o ue0 -j MASQUERADE $IPTABLES -A FORWARD -i ue0 -o ue1 -m state --state RELATED,ESTABLISHED -j ACCEPT $IPTABLES -A FORWARD -i ue1 -o ue0 -j ACCEPT #VPN $IPTABLES -t nat -A POSTROUTING -o tun0 -j MASQUERADE $IPTABLES -A FORWARD -i tun0 -o ue1 -m state --state RELATED,ESTABLISHED -j ACCEPT $IPTABLES -A FORWARD -i ue1 -o tun0 -j ACCEPT ---------------------------------------------------------------------- avgapon · 2022-06-14 09:12 Possible alternative solution: enable multiple FIBs, assign ue1 to fib1, point the default route of fib1 to tun0. ---------------------------------------------------------------------- Feelsnicy · 2022-06-14 09:46 ok and what will be the code for this to work? ---------------------------------------------------------------------- avgapon · 2022-06-14 09:58 Start with man or google. ---------------------------------------------------------------------- lazy-xo · 2022-06-18 11:19 Cmon - start with google? OP forgive my fellow old schooler - thought we learned in irc replying with RTFM and go google just makes people hesitant to ask for help. ---------------------------------------------------------------------- lazy-xo · 2022-06-18 12:20 OP - let’s walk through what you have and let’s fix this with zero use of google. I’m happy to help figure this out with you - google has provided results that harmed you. Yes fellow old schooler he did google and got results of it working on Linux and has even tested if they work. ::- I think what you’re looking for and want to accomplish is simply a routing issue. Can you run netstat -r and share the output? ---------------------------------------------------------------------- Feelsnicy · 2022-06-18 15:08 Sure, i already googled, read ipfw man and usage examples but didnt find similar setups also asked same question on other [unix forum](https://forums.freebsd.org/threads/route-all-traffic-to- vpn.85483/#post-571638) not yet solved. I also have been suggested to use fibs, other guys says no need. Regarding your netstat question. That one before starting openvpn connection (not using ipv6): ​ `# netstat -nr4` `Routing tables` `Internet:` `Destination Gateway Flags Netif Expire` `default 192.168.1.1 UGS ue0` `127.0.0.1 link#1 UH lo0` `172.16.1.0/24 link#3 U ue1` `172.16.1.1 link#3 UHS lo0` `192.168.1.0/24 link#2 U ue0` `192.168.1.15 link#2 UHS lo0` ​ ​ After starting openvpn connection `# netstat -nr4` `Routing tables` `Internet:` `Destination Gateway Flags Netif Expire` `0.0.0.0/1 10.8.0.1 UGS tun0` `default 192.168.1.1 UGS ue0` `10.8.0.0/24 10.8.0.1 UGS tun0` `10.8.0.1 link#4 UH tun0` `10.8.0.3 link#4 UHS lo0` `127.0.0.1 link#1 UH lo0` `128.0.0.0/1 10.8.0.1 UGS tun0` `172.16.1.0/24 link#3 U ue1` `172.16.1.1 link#3 UHS lo0` `178.30.200.90 192.168.1.1 UGHS ue0` `192.168.1.0/24 link#2 U ue0` `192.168.1.15 link#2 UHS lo0` ​ ​ 178.30.200.90 external ip address obtained via tun0 192.168.1.1 ue0 gateway ip connected to Internet 172.16.1.1 ue1 internal lan ip ---------------------------------------------------------------------- lazy-xo · 2022-06-18 17:27 I figured you’d googled and then some.. -) Can you share your ifconfig output as well? If we go to the basics when you say one for internet and one for lan clients and vpn -it’s something like this and want two client networks one going through this invisible tunnel: clients (172.16/) >> [[ IPFPI ]] >> INTERNET clients (192.168.1/)) >> [[ IFPI+TUNNEl ]] INTERNET Not clients (172.16/) >> [[ IFPI+TUNNEl ]] INTERNET —— Correct? Can you assign static addresses to something connected to ue_0 for the 172.16/ and trace route / arp -a and do the same when the tunnel is active? ---------------------------------------------------------------------- lazy-xo · 2022-06-18 17:41 Oh - and any if_bridge configurations - ---------------------------------------------------------------------- Feelsnicy · 2022-06-18 19:48 Yes, two networks going through one tunnel tun0 (VPN connection). In my representation it looks like: ISP<===>tun0<==INTERNET clients(192.168.1/)<== ===>clients(172.16/) and if VPN not running: ISP<===>INTERNET clients(192.168.1/)<== ====>clients(172.16/) `# ifconfig -a` `lo0: flags=8049 metric 0 mtu 16384` `options=680003` `inet6 ::1 prefixlen 128` `inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1` `inet 127.0.0.1 netmask 0xff000000` `groups: lo` `nd6 options=21` `ue0: flags=8843 metric 0 mtu 1500` `options=80009` `ether b8:27:eb:9f:19:56` `inet 192.168.1.15 netmask 0xffffff00 broadcast 192.168.1.255` `media: Ethernet autoselect (100baseTX )` `status: active` `nd6 options=29` `ue1: flags=8843 metric 0 mtu 1500` `options=80008` `ether 34:76:c5:08:0b:3f` `inet 172.16.1.1 netmask 0xffffff00 broadcast 172.16.1.255` `media: Ethernet autoselect (none)` `status: no carrier` `nd6 options=29` `tun0: flags=8051 metric 0 mtu 1500` `options=80000` `inet 10.8.0.3 --> 10.8.0.1 netmask 0xffffff00` `inet6 fddd:1194:1194:1194::1001 prefixlen 64` `inet6 fe80::ba27:ebff:fe9f:1956%tun0 prefixlen 64 scopeid 0x4` `groups: tun` `nd6 options=21` `Opened by PID 1200` p.s. Sorry hard to insert original formatting here. ---------------------------------------------------------------------- lazy-xo · 2022-06-19 06:27 Ok - this isn’t exactly my best diagram but working with what I have on a phone at a bar. Hopefully it’s enough to make sure I’ve got it based on the ifconfig output. You have to interfaces and ue_1 is the 172.16.1.0/24 the other ue_0 are the 192.168.8.1/24 and your tunnel is a 10.0.0.1/8 network punching through the isp and back (had to use two of the up and down little networks). Sorry a bit tipsy but but is the picture incorrect and should be more like this one here in a sec [IPFPI - config A](https://ibb.co/bB4GrmP) Or this [IPFPI - config B](https://ibb.co/8XG1Bdf) — either way we got this and without google just liquor and why do my images show up ---------------------------------------------------------------------- lazy-xo · 2022-06-19 06:46 Oh what does trace route and ipfw show when you switch the tunnel on from a clients on each of the nets? We’ll have this done in a few more exchanges after helping me understand the config a or config b either one ---------------------------------------------------------------------- Feelsnicy · 2022-06-19 07:41 The most close it is **IPFPI-config A** picture traceroute from Freebsd box `# traceroute 8.8.8.8` `traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 40 byte packets` `1 10.8.0.1 (10.8.0.1) 140.702 ms 139.986 ms 139.450 ms` `2 178.32.216.253 (178.32.216.253) 140.198 ms 140.270 ms 140.597 ms` `3 10.17.50.66 (10.17.50.66) 140.441 ms 140.272 ms` `10.17.50.64 (10.17.50.64) 140.200 ms` `4 10.73.16.112 (10.73.16.112) 140.390 ms` `10.73.16.116 (10.73.16.116) 139.801 ms` `10.73.17.70 (10.73.17.70) 140.175 ms` `5 10.95.64.142 (10.95.64.142) 179.006 ms` `10.95.64.0 (10.95.64.0) 140.835 ms` `10.95.64.140 (10.95.64.140) 214.219 ms` `6 par-gsw-sbb1-nc5.fr.eu (54.36.50.228) 143.800 ms` `par-th2-sbb1-nc5.fr.eu (54.36.50.226) 143.735 ms` `par-gsw-sbb1-nc5.fr.eu (54.36.50.228) 143.820 ms` `7 10.200.2.69 (10.200.2.69) 143.963 ms` `10.200.2.75 (10.200.2.75) 144.339 ms` `10.200.2.69 (10.200.2.69) 144.446 ms` `8 * * *` `9 * * *` `10 dns.google (8.8.8.8) 144.137 ms 143.912 ms 143.965 ms` traceroute from one of client connected to ue1 `% traceroute 8.8.8.8` `traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets` `1 172.16.1.1 (172.16.1.1) 1.947 ms 1.044 ms 1.029 ms` `2 * * *` `3 * * *` `4 * * *` `5 * * *` `6 * * *` `7 *^C` ---------------------------------------------------------------------- Feelsnicy · 2022-06-19 09:45 `# ipfw show` `00100 12 600 allow ip from any to any via lo0` `00200 0 0 deny ip from any to 127.0.0.0/8` `00300 0 0 deny ip from 127.0.0.0/8 to any` `00400 0 0 deny ip from any to ::1` `00500 0 0 deny ip from ::1 to any` `00600 4 380 allow ipv6-icmp from :: to ff02::/16` `00700 0 0 allow ipv6-icmp from fe80::/10 to fe80::/10` `00800 1 76 allow ipv6-icmp from fe80::/10 to ff02::/16` `00900 0 0 allow ipv6-icmp from any to any icmp6types 1` `01000 0 0 allow ipv6-icmp from any to any icmp6types 2,135,136` `65535 93872 22420969 allow ip from any to any` ---------------------------------------------------------------------- lazy-xo · 2022-06-19 14:46 Ok - now you’re cooking - let’s go back to your route tables and like ipfw look at the first output and then the next and back down to the ipfw show. Where do you see traffic not being managed the way you want? Let’s go through from top to bottom - route one output and route two output - each interface from the ifconfig output (I’ve been doing the same and suggest separating each one and details matter when you look at each - I put them into tables just to have a way to put the options and groups into cells) Then your trace route and look at the traffic that’s there ipfw is dealing with and what it’s not - Lastly your firewall rules go down like ipfw starting at 100 and why not much seems to be a match till the any any allow (ignore anyone that says how default deny absolutely has to be the policy - we ignore policy and follow rules). —- If you just want to have your tunnel I can replicate the IPFPi and get it to you but betting you’re more about the flow of packers and frames than the invisible tunnel. ---------------------------------------------------------------------- Feelsnicy · 2022-06-19 19:55 I do understand how to enable simple natd. And it works but if no vpn connection active. `#ipfw -q add divert natd ip from any to any via ue0` ipfw show then looks like: ​ `# ipfw show` `00100 0 0 allow ip from any to any via lo0` `00200 0 0 deny ip from any to 127.0.0.0/8` `00300 0 0 deny ip from 127.0.0.0/8 to any` `00400 51 4325 divert 8668 ip from any to any via ue0` `00500 51 4325 allow ip from any to any` `65535 173302 40255294 allow ip from any to any` Then if I enable openvpn connection, ue0 clients getting internet via tun0 and ue1 loosing internet connection. From that point eu1 clients lack forwarding to tun0. ---------------------------------------------------------------------- lazy-xo · 2022-06-20 05:47 Oh no doubt - see that you do - and you’ve already kinda figured it out. Let’s back up a bit to your firewall.conf: You have to physical interfaces and we have : three networks of our own - 10/8 - 172.16/16 - 192.168.1/24 and some routes to other networks (isp and a tunnel through the isp). in out / trust untrust / wan lan zones / are for people who like walls and - not the types who tend to gravitate to ipfw - your pi is definitely not walled thinking :) So you pointed out you get NAT (yep) and then hit the nail on the head with it stops forwarding. Why? You have a rule (400) that diverts everything, and it worked with iptables (but look close at how they’ve spaghetti confusing groups of dash -wtf and caps POSTROUTING). I have no idea what that means because both groups end up forwarding (routing right?). Here’s the good news - We have two physical interfaces but —- we can create as many virtual interfaces as we want they just need a parent.. we can create virtual bridges and we can also create virtual networks and routes to and from em. I can replicate what you’re wanting to do like I said - I can’t pin point where precisely for your IPFPi (don’t have all your rules and there’s more than one way to skin a cat). But we’re almost there and I wanna see it (no google tops it off). Remember we can create interfaces / networks and provide routes between them (default *gateway* is nothing but a route to send traffic arriving to ipfw wanting to go somewhere and cant be sent through an (static) assigned route (gateway just not the default). ---------------------------------------------------------------------- lazy-xo · 2022-06-20 06:02 Also - if you see the amount of traffic making it to the default selection? To make things easier to troubleshoot you can create any any rules for specific ports protocols source destination etc and see see what ipfw is matching to - for instance begin working back from 65534 and use the network names you’ve got and create ( add allow tcp from lan to wan via ue0 then udp and split 6 and 4) You can’t get too granular with your rules while you’re doing this and having some broken out is good practice anyway so if there’s an issue where you want deal with a protocol or something you can isolate it (or stop something all together without all traffic stopping) -default rule should have nothing matching and logged so you can identify where and how it didn’t match - after you solve this you can create rules for dns web mail and so on to allow and match - put the traffic that’s seen the most higher so it’s matched (you don’t want ipfw matching web or dns as one of the last rules). -use the names you assigned and they don’t have to be lan wan - ipfw will take from an add that has me just fine on it’s own;) ---------------------------------------------------------------------- lazy-xo · 2022-06-20 06:21 Sorry most important - none of your rules are to deny or reject or drop and yet the 172 client doesn’t go anywhere when you trace-route - start with naming that network and what interface(s) ipfw should know know about ---------------------------------------------------------------------- Feelsnicy · 2022-06-23 09:37 The working ipfw code below. No need enable natd, no need second fib. `${ipfw} nat 1 config log if tun0 reset same_ports deny_in` `${ipfw} add allow ip from me to any via ue0` `${ipfw} add allow ip from any to me via ue0` `${ipfw} add deny ip from 172.16.1.0/24 to any via ue0` `${ipfw} add deny ip from any to 172.16.1.0/24 via ue0` `${ipfw} add nat 1 ip from any to any via tun0` Rule sets after applying `# ipfw show` `00100 703 122599 allow ip from me to any via ue0` `00200 769 254365 allow ip from any to me via ue0` `00300 0 0 deny ip from 172.16.1.0/24 to any via ue0` `00400 0 0 deny ip from any to 172.16.1.0/24 via ue0` `00500 1004 281632 nat 1 ip from any to any via tun0` `65535 1847 448672 allow ip from any to any`