Post B50TVnuwMTHnrvaWzg by azonenberg@ioc.exchange
(DIR) More posts by azonenberg@ioc.exchange
(DIR) Post #B50LcEY5SamgZKSBou by azonenberg@ioc.exchange
0 likes, 0 repeats
Since coming back from Germany for HARRIS I've been chasing a weird issue on my ThunderScope 100baseT1 test setup.Before leaving for HARRIS I unplugged the TS, left the rest of the setup untouched, and brought the TS with me to demo to people while traveling.After coming back the TS appears fine (passed calibration, and gives good-looking eyes on 100baseTX).But when hooked back to the same, theoretically unmodified baseT1 test setup, I get weird diagonal distortion, lots of what I think is mostly common mode noise that bleeds a bit into differential mode too, and eye closure.Before and after. Same ngscopeclient version, same session file, same DUT, same scope. What changed?
(DIR) Post #B50LqIBYjgtJtycVlo by azonenberg@ioc.exchange
0 likes, 0 repeats
I tried swapping one of the two media converters at a time (I only have a single third unit to test with). The third one had been sitting in my "to fix" bin for a while with a ripped-off USB power jack but I reworked it and got it working. No change in observed behavior.Other things I've already tried: disconnect the laptop from the mains supply, disconnect the baseT connections from the media converters leaving the T1 link up, power the media converters off the laptop USB ports instead of a separate device on the bench that was closer.Nothing seems to have any effect.I also decabled the entire setup, jiggled a bunch of stuff around, and redid all of the twisted pair connections to the screw terminals after power cycling the media converters. No change.
(DIR) Post #B50M9UpPxDqqcCQ96u by azonenberg@ioc.exchange
0 likes, 0 repeats
Next up: swapping the SMA cables (Mini-Circuits FL086-12SM+, not manufacturer serialized but marked with my internal tracking IDs 0056, 0057, 0058, 0064) from the directional coupler to the ThunderScope.I haven't VNA tested them but I've quarantined the ones that were on the bench setup. If new cables creates a change in behavior, I'll test them individually to see which was at fault. If no change they'll go back in general lab inventory.
(DIR) Post #B50NWo3oiP06sSppLs by azonenberg@ioc.exchange
0 likes, 0 repeats
Nope, looks about the same
(DIR) Post #B50Nem8rjSZ7FAzpo0 by azonenberg@ioc.exchange
0 likes, 0 repeats
@SteveSyfuhs Lab is climate controlled and is a pretty stable interior environment... 41% RH right now (fairly typical for the last few weeks).Temperature has been 72 +/- 1 F near the server racks for the past 24 hours according to the thermometer I have there, the test setup is a little closer to the heat pump and is slightly cooler as a result.
(DIR) Post #B50OINboMhRAjsQ7GK by azonenberg@ioc.exchange
0 likes, 0 repeats
Next, let's try shortening the UTP runs to see if they're coupling noise from something nearby into the DUT.Shortening the orange leg seems to have improved things a bit?
(DIR) Post #B50OPUcQhUxyVV6Kx6 by dragosr@chaos.social
0 likes, 0 repeats
@azonenberg Since you've eliminated most of the signal chain, have you checked what's physically adjacent to the baseT1 fixture? Anything new or repositioned near the test setup? A switched-mode PSU, active USB hub, monitor, could be coupling into the single pair path. BaseT1 won't tolerate ambient EMI that baseTX shrugs off. Try powering down everything nearby that isn't part of the measurement and see if the eyes clean up.
(DIR) Post #B50Ogj8xx1ZyTgXOwS by azonenberg@ioc.exchange
0 likes, 0 repeats
Shortened the green leg on the other side of the fixture doesnt seem to have helped much.One thing I did notice that doesnt show up well in still screenshots is the dynamics. It seems like the signal is jumping between multiple amplitudes which makes no sense.'Not like PAM, like the P-P amplitude is not constant.
(DIR) Post #B50OhgAmTv5VYOqBSi by LapTop006@aus.social
0 likes, 0 repeats
@azonenberg I assume you've already checked for environmental noise (bad new switcher too close etc.)?
(DIR) Post #B50OnUcCMkvPRgRAB6 by azonenberg@ioc.exchange
0 likes, 0 repeats
@dragosr That's what I'm trying to figure out. I unplugged the client device nearby that is the only reasonably plausible noise source in the area that is all new (but it was there before I left on the trip and things looked fine.I feel like I'm missing something obvious.
(DIR) Post #B50PGiYzaN1F6SsBkW by dragosr@chaos.social
0 likes, 0 repeats
@azonenberg AFAIR That diagonal smear pattern is characteristic of a reflection mixing with the signal (it's been a long time since I worked for a measurement equipment maker, memory fuzzy :-) ) Is there a directional coupler to swap too? If so have you done a VNA S-sweep on it? degraded solder joint could cause reflection.
(DIR) Post #B50PSNzDuqaylBhqVs by azonenberg@ioc.exchange
0 likes, 0 repeats
@dragosr The coupler is one of the few things I have not swapped. I'd be very surprised if it went bad but it's possible I guess.I don't have spares but I can source them from minicircuits. If I can't figure anything out in the next day or two I'll probably try that.
(DIR) Post #B50Q2vPn6gfcrUYXsu by dragosr@chaos.social
0 likes, 0 repeats
@azonenberg don't assume you have a four port VNA, but putting terminators on two and running S21 for insertion loss (input>through), S31 for coupling factor (input>coupled), S41 for isolation (input>isolated), and S11/S22/etc. for return loss on each port individually might show bad solder etc. with any deviations if you sweep dc->100Mhz or wider
(DIR) Post #B50Q3GNj9YUttAkh5E by azonenberg@ioc.exchange
0 likes, 0 repeats
The plot thickens.Unplugged power from both media converters but left everything cabled up as-is.I see the same noise spikes.Channels 1/2 (A direction) and 3/4 (B direction) are seeing approximately the same noise waveform, but with reversed polarity.
(DIR) Post #B50QEQ83Fn8HIwjwiu by azonenberg@ioc.exchange
0 likes, 0 repeats
The overall noise waveform appears to be ~2.8 ms off, ~2.9 ms of stronger noise, and ~2.3 ms of the strongest noise then repeating.The noise bursts consist of 38 kHz repetition pulses with ~356 kHz narrower pulses between them. The highest frequency noise is around 4 MHz.
(DIR) Post #B50QMcF26SYtBhuh8K by azonenberg@ioc.exchange
0 likes, 0 repeats
Unplugged channels 3 and 4 of the TS from the fixture and I no longer see the noise bursts on those channels.So it's not the scope doing it. I'm measuring real noise from somewhere.
(DIR) Post #B50QgoSSONDSrtp5sW by azonenberg@ioc.exchange
0 likes, 0 repeats
Unplugged the RJ45 from the right hand media converter (the long haul one going into the in-wall cable plant) and the noise got a lot smaller.Which is baffling because that RJ45 has no DC connection to anything in the DUT, it's just going into a magjack which drives the baseT PHY.
(DIR) Post #B50QwoepPyXc7XEysy by azonenberg@ioc.exchange
0 likes, 0 repeats
Noise waveform with the media converters powered off but the RJ45 connected
(DIR) Post #B50R6K9hg8DCPrDSHg by azonenberg@ioc.exchange
0 likes, 0 repeats
like seriously how am I getting 40 mV p-p of noise *through the directional coupler with 10 dB coupling loss* coming out of my baseT1 PHY when the board isnt even powered up
(DIR) Post #B50Ro9ijpvd8nJVIbg by azonenberg@ioc.exchange
0 likes, 0 repeats
The noise seems to be pervasive. Unplugging the RJ45 from the wall jack but leaving it plugged into the media converter reduces it.With the patch cord connected to the media converter and not the wall jack I see 16 mV p-p, with it plugged into the wall I see 35 mV.Turning off the HVAC or ultrasonic cleaner had no discernable effect.
(DIR) Post #B50RrqQZVeAO7HlWaG by lasse@social.tchncs.de
0 likes, 0 repeats
@azonenberg is your PV inverter working right now?
(DIR) Post #B50RwER0C681jO3i1A by astraleureka@social.treehouse.systems
0 likes, 0 repeats
@azonenberg get the shotgun, DUT's haunted
(DIR) Post #B50RztSc7JITq7YhXs by jpm@aus.social
0 likes, 0 repeats
@azonenberg crosstalk? Can happen in dense patching
(DIR) Post #B50S0D3pFBLud27BgW by azonenberg@ioc.exchange
0 likes, 0 repeats
Unplugging the wall side RJ45 from the media converter leaving the laptop connected drops the noise to ~8 mV.Removing the RJ45 from the other media converter actually makes it get slightly... worse? that makes no sense. but probably within noise floor of my measurements
(DIR) Post #B50S1uubguxgv8cK24 by azonenberg@ioc.exchange
0 likes, 0 repeats
@lasse it's almost 10pm local time so should not be producing, although they do talk to the controller over powerline comms so there might still be data traffic
(DIR) Post #B50SAkxO4cnWeS8HRo by azonenberg@ioc.exchange
0 likes, 0 repeats
OK so now we have a couple of questions.1) What is the root noise source? Why is it new, what changed from a few weeks ago?2) How the hell is it coupling through my unpowered media converter and into my test setup3) What can I do to get rid of at least the observed noise at my DUT?
(DIR) Post #B50SUkQdkNOCnyLdYm by azonenberg@ioc.exchange
0 likes, 0 repeats
Chassis ground of the RJ45 on the media converter is connected to circuit ground. It's a Bel Fuse 0826-1G1T-23-F.The RJ45 and cable are unshielded, so there is no connection from a cable shield to the metal shell of the board-side RJ45.But the Bob Smith termination on the cable is connected to that same shield connection. I have to assume this is how the noise is coupling in.
(DIR) Post #B50Si8UvZNzTDcjmTY by azonenberg@ioc.exchange
0 likes, 0 repeats
The noise is *strong*. I just plugged a ZS1500 active probe into my LeCroy scope planning to use it to measure the actual voltage on the media converter board.With the probe just sitting on my lab bench, not doing anything, it sees almost 200 mV p-p with a very similar looking waveform.
(DIR) Post #B50SyU7ExjXmtnSBKi by ldcd@social.treehouse.systems
0 likes, 0 repeats
@azonenberg I've gotten fairly strong pickup from monitors before, especially older ones
(DIR) Post #B50SyUIEIsKnRsaxSi by azonenberg@ioc.exchange
0 likes, 0 repeats
@ldcd except I don't have any new monitors, none of this stuff is new.The server/workstation upgrades I did a while back are the only things I can think that might have been involved??
(DIR) Post #B50T31JLU9Bz0iHJiK by azonenberg@ioc.exchange
0 likes, 0 repeats
Noise waveforms seen by ZS1500.They get stronger when the probe is sitting flat on the steel lab bench and weaker when it's lifted up on top of something.
(DIR) Post #B50TA3cDJfLgBUSXei by johnlogic@sfba.social
0 likes, 0 repeats
@azonenberg That's starting to sound like you're picking up radar, like the kind used off the San Francisco coast to monitor ocean wave heights. I'd sometimes pick that up on HF around/below 4 MHz. Your building wiring is probably a good antenna for it.
(DIR) Post #B50THogQiVWUBr8CVU by azonenberg@ioc.exchange
0 likes, 0 repeats
@johnlogic ... oh FFSVarious sites of naval base Kitsap aren't too far away. If they started firing up air defense radars around the base or on ships in response to the Iran situation, I might be picking that up...
(DIR) Post #B50TLuB0EfJ4NwNgmm by azonenberg@ioc.exchange
0 likes, 0 repeats
@johnlogic but I would expect that to be C/X band or similar microwave stuff. Maybe i'm picking up subharmonics of it or something? or PIM?
(DIR) Post #B50TSEl2cz3NxEG0Nk by gsuberland@chaos.social
0 likes, 0 repeats
@azonenberg do you have a current going through your bench? this might explain some of it.
(DIR) Post #B50TVU1eJolkAwZbfs by johnlogic@sfba.social
0 likes, 0 repeats
@azonenberg check for coastal wave radar in your area. (Do you have/know where you can find a nearby HF rig?)
(DIR) Post #B50TVnuwMTHnrvaWzg by azonenberg@ioc.exchange
0 likes, 0 repeats
@gsuberland I shouldn't, lol.
(DIR) Post #B50TXqiwti9h9VxFR2 by gsuberland@chaos.social
0 likes, 0 repeats
@azonenberg shouldn't, buuuuuut...
(DIR) Post #B50TZLnmpsHfpRvvO4 by dragosr@chaos.social
0 likes, 0 repeats
@azonenberg i thinkkthe steel bench data is a smoking gun - environmental - 200 mV p-p just sitting on the steel bench — stronger flat, weaker elevated — confirms the bench itself is conducting or re-radiating the interference. The steel surface is coupling magnetically or capacitively to whatever the source is.38 kHz is a common switching frequency for LED drivers and small power converters. Walk around with probe and find what's radiating?It's a fox hunt.
(DIR) Post #B50VEJNcqScQ90nAvo by dragosr@chaos.social
0 likes, 0 repeats
@azonenberg coupling: termination > chassis ground > cable plant acting as antenna. The unpowered magjack still passes this because it's a passive magnetic coupling = no power needed.
(DIR) Post #B50VPfGBpsyXBojB56 by azonenberg@ioc.exchange
0 likes, 0 repeats
@dragosr yes but the phy on the far side is an active device. Is it coupling into media converter ground then into the tx signals that way? I can't fix it until i understand the exact path of the interference
(DIR) Post #B50VwI5V0lSGsUHhmS by dragosr@chaos.social
0 likes, 0 repeats
@azonenberg I think wall cable acts as antenna, picks up the 38 kHz common mode on Ethernet, passes through the magjack center taps into Bob Smith and onto media converter' ground. BaseT1 PHY TX references same ground plane for output. So when ground bounces at 38 kHz, the TX output rides with it. Modulated signal travels baseT1 link through your directional coupler into scope. The PHY being active converts ground bounce into signal-referenced noise.
(DIR) Post #B50WJcIG3KqgCnNJQG by ChuckMcManis@chaos.social
0 likes, 0 repeats
@azonenberg Don't suppose you have EMC probes do you? That could give you the polarization of the noise which might help identify the source. If you rotate the probe so it is pointed vertically, does the noise change? Next a piece of grounded foil (or plated PCB) above the probe? And then below the probe. The biggest culprits in my lab seem to be cheap USB chargers and/or LED bulbs.
(DIR) Post #B50f1OpMHQEmoeFIGG by moonwick@mastodon.energy
0 likes, 0 repeats
@azonenberg 2.8 + 2.9 + 2.3 ms is awfully close to 120Hz, this smells a lot like it's coming from an AC/DC somewhere. If you get an isolated probe and hook that up to L-N does the noise look like it's in phase (and stays in phase) with the grid?Also was this during daylight or after dusk? Just thinking about all of those microinverters you added recently...
(DIR) Post #B50gFO0EF77uGaYzku by tnt@chaos.social
0 likes, 0 repeats
@azonenberg I know a couple of years back I had a similar issue ( went to a trip, came back and got weird noise on my scope ). Turned out to be a switching PSU brick that had failed. Completely unrelated to the test setup, just happened to be on the same circuit as my bench and coupling noise somehow.And the PSU was still working, like the device it powered was still up and working. I had to go through my whole appartement unplugging thing one by one to find where it was coming from.
(DIR) Post #B51JJp83E7FPleKTBY by 0h00000000@ioc.exchange
0 likes, 0 repeats
@azonenberg 38kHz is the sweet spot for MOSFET power switching, could be some power supply that skips cycles. I had a similar issue and it took me a while to figure out it was the power supply for the microscope LED lights.