[HN Gopher] Apple Patent Shows GPU Dynamic Caching Has Been in D...
___________________________________________________________________
Apple Patent Shows GPU Dynamic Caching Has Been in Development for
Years
Author : CharlesW
Score : 68 points
Date : 2023-11-02 18:43 UTC (4 hours ago)
(HTM) web link (www.tomshardware.com)
(TXT) w3m dump (www.tomshardware.com)
| skavi wrote:
| Anything shipping in a chip has to have been developed for years.
| And this particular GPU arch was meant to ship last year.
| baggy_trough wrote:
| Would love to read a good explanation of what this is.
| jb1991 wrote:
| It describes it right in the article my dude.
| skavi wrote:
| Personally wouldn't consider that a good explanation. Would
| love if more details were released.
| yunohn wrote:
| I'm in the same boat, what exactly enables this dynamism
| that couldn't be achieved before? I'm missing the reason
| why this was static up until now.
| slaymaker1907 wrote:
| Given what I know about CUDA and assuming it's different from
| being able to call malloc within a kernel and paging memory
| between the CPU and GPU, I assume it's adding in some smartness
| for offloading stuff from the GPU when running multiple
| kernels. You want to keep stuff in GPU memory as much as
| possible since transfers can increase latency dramatically, but
| if you have multiple applications using the GPU then there
| might not be enough memory for all of them.
|
| However, I'm always suspicious of how innovative patents
| actually are so it wouldn't surprise me to find out it's just
| barely different than what CUDA or unified memory on game
| consoles can do.
| bangonkeyboard wrote:
| All Apple Silicon is unified memory.
| stefanfisk wrote:
| What about https://www.digitaltrends.com/computing/apple-
| dynamic-cachin...?
| tedunangst wrote:
| Even after reading that, I'm still uncertain how it lets more
| blocks/waves/whatev execute. Could really use an example with
| some fictitious numbers.
| kimixa wrote:
| Yeah, vague terms like "local memory" aren't helpful -
| there's lots of different memories that are local to the
| shader unit that are a shared resource so can affect
| occupancy (the LDS, register file, TBDR stuff like the tile
| buffer, and probably any number of other things). Which of
| these is it referring to? They are separate things in some
| architectures, but maybe not in all?
|
| There's the linked patent itself [0] - but that specifically
| seems to refer to using the MMU as the part that's doing this
| dynamic allocation and translation - but the majority of
| those resources above are "before" address translation, with
| the MMU often being more at the L2 cache level rather than
| embedded within the shader clusters themselves. Maybe this
| isn't the "normal" MMU and instead a simpler address
| translator specifically for those resources that has been
| added of for this? Or is it something like the parameter
| buffer (The block of memory used to store the intermediate
| data between the tiling & rasterization state, and the pixel
| shaders). That has been "dynamically allocated" in a similar
| way since Apple were just taking PowerVR cores directly.
|
| And then what is the cost of this re-allocation? If there's a
| cost to performance to allocating new pages, or (like worse)
| running out of spare pages, it might mean the (graphics API)
| user still has to be aware of their resource usage in
| scheduling shaders, so less of the promise of "Just throw
| things at the hardware and it'll do things optimally" than
| people might hope.
|
| [0] https://patents.google.com/patent/US20210271606A1/en
| nxobject wrote:
| Real World Tech's mailing list will have a good conversation
| about it at some point, for sure. (Although I know the
| community isn't primarily about GPU.)
| AaronFriel wrote:
| Is macOS Dynamic Caching similar to Windows hardware-accelerated
| GPU Scheduling[1]? That feature purports improve latency and GPU
| scheduling efficiency. Is the feature here that the OS is
| delegating more scheduling to the M3's ASC coprocessor[2] -
| assuming it's similar to the M1?
|
| It seems to me that every explanation of dynamic caching in terms
| of memory is "wrong" - as seen here and in several articles
| written by folks more familiar with PC hardware.
|
| I think where some folks might get it wrong is thinking of Apple
| silicon as being like PC hardware, where VRAM and RAM are
| distinct pools of memory and using the CPU to move data between
| them (or other devices) is very inefficient. PCI bus attached
| devices having separate pools of memory has given rise to a
| plethora of technologies to allow directly read from RAM (DMA),
| or to enable a GPU to read from NVMe (DirectStorage on Windows),
| and so on.
|
| The patent from the article seems to describe unified memory,
| part of the M1's architecture, not whatever "Dynamic Caching" is;
| but I'll admit Apple makes it a bit hard to understand what
| exactly is the case.
|
| There's only one person I'd trust to describe what this feature
| actually is - I'll wait for Asahi Lina to break down what dynamic
| caching is and whether this is a hardware or OS feature.
|
| [1] https://devblogs.microsoft.com/directx/hardware-
| accelerated-...
|
| [2] https://asahilinux.org/2022/11/tales-of-the-m1-gpu/
| userbinator wrote:
| _I think where some folks might get it wrong is thinking of
| Apple silicon as being like PC hardware, where VRAM and RAM are
| distinct pools of memory and using the CPU to move data between
| them_
|
| The patent basically describes using a page table to
| dynamically map pages of RAM between the GPU and CPU, something
| that Intel's integrated GPUs have been doing for 20 years.
| photonerd wrote:
| It's more than that. That's just shared memory. As you say,
| if it was just that they it wouldn't be noteworthy at all
| withinboredom wrote:
| If it quacks like a duck, looks like a duck, it's probably
| a duck.
| userbinator wrote:
| Unless it's from Apple, in which case they call it a
| peacock and everyone gets all hyped about it.
| userbinator wrote:
| _Apple 's GPU Dynamic Caching ensures that cache and memory
| spaces are dynamically assigned based on the actual needs of
| different tasks and workloads_
|
| Isn't this not too different from what Intel DVMT was doing
| twenty years ago?
|
| https://en.wikipedia.org/wiki/Dynamic_video_memory_technolog...
|
| Then again, this is the same Apple that took "integrated
| graphics" / "UMA" as a marketing point.
| foota wrote:
| I don't think this is correct. My understanding is that this is
| about more efficiently utilizing the GPUs available resources
| (e.g., cache) at the core-ish level. Like imagine if
| hyperthreads could share register space based on what they're
| doing at the time, I think.
| acchow wrote:
| The OP isn't really about UMA, but about caching.
|
| In any case, in Intel's DVMT you have a dedicated memory space
| for graphics. It is a dynamic amount, but it is still dedicated
| for graphics. You need to copy data from system memory over to
| graphics memory for it to be handled.
|
| In UMA, no data copying is needed. There's 1 memory space
| kcb wrote:
| Zero copy shared memory absolutely is available on all modern
| IGPUs. Do people really think such a low hanging fruit was
| ignored for the past decades?
|
| https://www.intel.com/content/dam/develop/external/us/en/doc.
| ..
|
| https://en.wikipedia.org/wiki/Heterogeneous_System_Architect.
| ..
| userbinator wrote:
| _You need to copy data from system memory over to graphics
| memory for it to be handled._
|
| No you don't. You can map any page of physical RAM into the
| GPU address space.
|
| This has been the case since at least the Intel i810. Here's
| a relevant part from the i815 documentation:
|
| https://i.imgur.com/p53Dxsn.png
| Jasper_ wrote:
| No, it's about replacing the static shader register allocation
| with more of a cache, allowing for more dynamic occupancy usage
| of the GPU. Memory is still memory.
| tedunangst wrote:
| The patent mentions MMU? Is Apple mapping the GPU register
| file into the CPU address space?
| lwkl wrote:
| These GPUs reserved a variable amount of your system memory for
| the GPU. So if you had 4 GB of RAM the GPU would maybe use 500
| MB leaving you with 3.5 GB of RAM for your CPU. There was often
| a option in the BIOS that allowed you to change the amount of
| memory reserved for the GPU.
| kcb wrote:
| That option is not a hard limit on the RAM IGPUs can use.
| lwkl wrote:
| There is a hard limit with DVMT that is described in the
| white papers by Intel. Let's say you allow up to 256 MB of
| RAM to be allocated for graphics. The memory could be
| partitioned in a way that 128 MB are fixed memory that are
| always only available for the graphics driver and 128 MB
| are DVMT memory that can be used by the OS if the graphics
| driver doesn't need it.
| clemlesne wrote:
| Hope Asahi team will integrate that soon.
| faitswulff wrote:
| The patent is here:
| https://patentimages.storage.googleapis.com/2d/72/df/117a401...
___________________________________________________________________
(page generated 2023-11-02 23:01 UTC)