Midi Packet Size is different between 32carbon and 64 bit cocoa lazarus / 33-macOS - Mac OS X ---------------------------------------------------------------------- Josh · 2019-06-19 21:24 Hi I have been battling with getting midi working with Cocoa. Code that works fine on Carbon; give incorrect data error on Cocoa. I have tracked down a problem; that I think is causing it; but do not know the best way to correct what I see as a BUG in the Cocoa Implementation of CoreMidi A midi Data Packet is a record timestamp:uInt64; length:uInt16; data:array[0..255] of byte The record structure has a size of 268bytes on carbon; where it shows the correct information; but on cocoa the exact same structure give a size of 272bytes. A similar issue exists with MidiPacketList which has 272(carbon) and 280 (cocoa). Could this be a memory allignment issue? Anyone have an idea of how to fix; as core midi and other mac frameworks exists; it is possible that the same could exists within these aswell. Hope the above makes sense. ---------------------------------------------------------------------- engkin · 2019-06-19 21:41 use packed record. ---------------------------------------------------------------------- Josh · 2019-06-19 22:21 Hi I have tried using packed Code: [Select]MIDIPacket = packed record timeStamp: MIDITimeStamp; length: UInt16; data: array [0..255] of Byte; end;                              But the size is still 272 bytes. ---------------------------------------------------------------------- engkin · 2019-06-19 22:25 Your first post shows: timestamp:uInt64; Your last post shows: timeStamp: MIDITimeStamp; Is MIDITimeStamp of uInt64 type? if not, replace it with uInt64. ---------------------------------------------------------------------- engkin · 2019-06-19 22:27 Wait!! ---------------------------------------------------------------------- engkin · 2019-06-19 22:32 That was not packed, sorry. Try this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$push}{$PackRecords 4}type  TMIDIPacket = record    timestamp:uInt64;    length:uInt16;    data:array[0..255] of byte;  end;{$pop} This should be 268 bytes. Packed would be 266 bytes. ---------------------------------------------------------------------- Josh · 2019-06-19 23:03 hi Code: [Select]{$push} {$PackRecords 4} MIDIPacket =  record timeStamp: UInt64; length: UInt16; data: array [0..255] of Byte; end; {$pop}      The above code has no effect on cocoa; it is still 272bytes... ---------------------------------------------------------------------- engkin · 2019-06-20 01:18 Probably your target is 64bit. Back to packed records: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  MIDIPacket = packed record    timestamp:uInt64;    length:uInt16;    padding:array[0..1] of byte;    data:array[0..255] of byte;  end; ---------------------------------------------------------------------- Thaddy · 2019-06-20 05:28 The last is correct indeed. Midi streams need packed records and {$packrecords 1} not 4! There can not be or should not be any slack spacing. This has nothing to do with alignment, but with reading/writing the stream correctly. Stored data should always be packed and Midi is no exception. IOW: byte alignment of the members. The in-memory record start itself will always be aligned to current alignment so there is no speed disadvantage in any way. ---------------------------------------------------------------------- Josh · 2019-06-20 09:23 Hi Something odd is going on, on my end. If I create a midipacket type I my application unit; with pack record 4, then the structure size is correct at 268. If I use unit MACOSALL; which uses MIDISERVICES definition, then the size is incorrect. If I modify the MIDIServies definition it still is not correct. Now if I make a syntax error in the MIDIservices definition; expecting a compile error; it does not and still compiles. So I then done a clean all build; and still it compiles . So question now is how can I modify midiservices. I have attached a project that show the issue, you can comment out the define statement to use either macosall definition or its own. Hopefully someone can replicate. ---------------------------------------------------------------------- Thaddy · 2019-06-20 10:16 Josh, read my explanation above: $packrecords 4 is wrong. You MUST use 1, which is the default. Think!!!!!! (note I am an avid audio and midi programmer) Record packing does not influence alignment of the record itself. ---------------------------------------------------------------------- Josh · 2019-06-20 10:40 Hi Thaddy, With carbon, the midipacket in macosall;midiservices is a size of 268. So for it to work with the same in cocoa and carbon the midipacket size must be the same ie 268. packrecords 1 create a size of 266  which is wrong packrecords 4 creates a size of 268 packrecords 8 creates a size of 272 without packrecords a size of 272 If I use Align 1; i still get 266 I see similar problem with midipacketlist type as well.. If the size is not the same in carbon and cocoa should it be placed in bugtracker; as the issue is with coremidi implementation. ---------------------------------------------------------------------- rvk · 2019-06-20 11:03 According to this the MIDIPacket and MIDIPacketList should have the 4 byte alignment. QuoteSetting {$Align 1} for the whole file was not the correct solution. Instead only the two records (MIDIPacket and MIDIPacketList) should have 4 byte alignments instead of 8 byte alignment.https://stackoverflow.com/a/37748973/1037511 QuoteNote the #pragma pack(push, 4) above the definition of MIDIPacket in the CoreMIDI.h header file. ---------------------------------------------------------------------- Thaddy · 2019-06-20 13:36 I am sorry but that is the wrong answer. The alignment is 1, not 4. I am writing music software for the past 20 years. There's no padding. In Freepascal the record in-memory will be properly aligned. The stored records are packed to 1. The reason that 4 works too is because that is the minimum size of the record header members. See also: https://en.wikipedia.org/wiki/MIDI#Technical_specifications Just to be sure: the midi format is completely platform independent. That "approved"  stackoverflow entry is simply wrong! You can test that by simply opening a midi file as a stream. Which, btw, is its intended use. (some people even stole my code - the reverb - without proper representation of its origins: Jezar's Freeverb 3, translated and optimized by me and stolen by Christian!, that's not midi, but... he still did not correct that even after asking for it politely. That's OK TobyBear and me know who wrote it...Stays freeware anyway... But I still hold a grudge because of misrepresentation, so Christian if you read this, put it right after all those years) ---------------------------------------------------------------------- skalogryz · 2019-06-20 14:29 Quote from: josh on June 20, 2019, 11:23:26 amSomething odd is going on, on my end. If I create a midipacket type I my application unit; with pack record 4, then the structure size is correct at 268. If I use unit MACOSALL; which uses MIDISERVICES definition, then the size is incorrect. If I modify the MIDIServies definition it still is not correct. ... So question now is how can I modify midiservices. It's not wrong or odd, but expected. If you change the sources of FPC packages (and in this case you do) you need to recompile AND reinstall the modified packages. (if I remember correctly, it should as easy as entering fpcsource/packages/unvint and running a command: make clean all install BUT depending on where your sources are (need of root access?) it might be a bit trickier. "install" is likely to require you the root access) Whenever you run a compiler it doesn't check the package source, instead it goes straight to recompiled binaries. (Btw, Lazarus can recompile a "Lazarus package" automatically, but it cannot recompile "FPC packages" automatically. And both MacOSAll and MIDIServices are FPC packages) ... and here's the project management problem. Even if you provide the patch (and I think you should) to correct MIDIServices declaration, you will not get the fixed version until the next official FPC release (which will come with the correctly compiled header). ---------------------------------------------------------------------- Josh · 2019-06-22 06:51 Hi Thanks everyone for their input. I managed to rebuild the modification and everythiong is now working correctly. I have created a bug tracker entry https://bugs.freepascal.org/view.php?id=35750 ---------------------------------------------------------------------- Thaddy · 2019-06-22 08:42 I have repeated my comments on the bug tracker. (And I checked the record sizes on my mini Mac) The pack size is wrong: that is programmer error and not a bug. Again: midi is a streaming protocol and does not allow slack space. Record packing should be byte aligned. Always. < RTFM  > What you seem to forget is that natural alignment between 32 and 64 bitness is not equal. ---------------------------------------------------------------------- BeRo · 2021-01-02 05:49 Quote from: Thaddy on June 22, 2019, 10:42:02 amI have repeated my comments on the bug tracker. (And I checked the record sizes on my mini Mac) The pack size is wrong: that is programmer error and not a bug. Again: midi is a streaming protocol and does not allow slack space. Record packing should be byte aligned. Always. < RTFM  > What you seem to forget is that natural alignment between 32 and 64 bitness is not equal. Sorry, but {$packrecords 4} is right here (or a packed record together with a AlignmentDummy UInt16 after the length UInt16). I've tested with my own pascal-native cross-platform Mini-DAW (Digital Audio Workstation), which I'm developing in the moment. You are probably confusing API data structures (here CoreMIDI) with the MIDI specification of the MIDI bytestream. ---------------------------------------------------------------------- Thaddy · 2021-01-02 06:29 No it is not.  It is 1. not 4. Midi is a streaming protocol. ---------------------------------------------------------------------- BeRo · 2021-01-02 15:58 Quote from: Thaddy on January 02, 2021, 07:29:10 amNo it is not.  It is 1. not 4. Midi is a streaming protocol. Sorry, but MidiPacket is a CoreMIDI data struct, only MidiPacket.data contains the raw MIDI Even in the original CoreMIDI header sources from Apple itself one can see "// MIDIPacket must be 4-byte aligned" as code comment. https://github.com/phracker/MacOSX-SDKs/blob/master/MacOSX10.9.sdk/Sys tem/Library/Frameworks/CoreMIDI.framework/Versions/A/Headers/MIDIServi ces.h#L1984 and see "#pragma pack(push, 4)" at https://github.com/phracker/MacOSX-SDKs/blob/master/MacOSX10.9.sdk/Sys tem/Library/Frameworks/CoreMIDI.framework/Versions/A/Headers/MIDIServi ces.h#L338 Josh is saying, Apple is saying it, I'm saying it. You are pretty much alone with your opinion, which is probably due to the misunderstanding on your part that the MidiPacket Struct contains completely only raw MIDI data, which it defacto does not. It is a container structure including header data for raw MIDI data. ---------------------------------------------------------------------- BeRo · 2021-01-02 16:01 (In addition, I work with it productively as a professional developer in the audio field as well.) ---------------------------------------------------------------------- BeRo · 2021-01-02 16:11 Code: C  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #include #include #include  #include #include #include #include  int main(){  int i = sizeof(MIDIPacket);  printf("%i\n", i);  return 0;}       Prints also 268 on my 2015 MacBook Pro 13 inch under a 64-bit MacOS Big Sur. ---------------------------------------------------------------------- freq · 2021-09-07 14:45 Quote from: josh on June 22, 2019, 08:51:21 amHi Thanks everyone for their input. I managed to rebuild the modification and everythiong is now working correctly. I have created a bug tracker entry https://bugs.freepascal.org/view.php?id=35750 Hi, can confirm that your solution with {$packrecords 4} is working also here with Intel Mac, 64bit under BigSur! Thanks for providing the solution! ---------------------------------------------------------------------- Key-Real · 2023-05-23 06:51 I have FPC Trunk 3.3.1 [2023/05/22] This Issue is still not solved I'm on an AArch64 Mac (M1) this works (around line 573): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$push}{$packrecords 4}  MIDIPacket = record                timeStamp: MIDITimeStamp;                length: UInt16;                data: array [0..255] of Byte;               end;{$pop} and (around line 612): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$push}{$packrecords 4}  MIDIPacketList = record                    numPackets: UInt32;                    packet: array [0..0] of MIDIPacket;                   end;{$pop}  pls fix