[HN Gopher] ESE: The server-grade, embedded DB ships with Windows
___________________________________________________________________
ESE: The server-grade, embedded DB ships with Windows
Author : mnkypete
Score : 64 points
Date : 2021-01-30 14:16 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| fanf2 wrote:
| IIRC this storage engine used to have a 2GB limit, but it did not
| properly check this limit and would corrupt itself if you tried
| to store too much. This caused problems for Exchange, Outlook,
| the ironically named Visual Source Safe, and probably others.
| brettsh wrote:
| This engine was never storage for Outlook, nor I _think_ Visual
| Source Safe (but can't be sure of that). The JET Blue engine
| often gets confused with the JET Red engine, which were both
| completely different implementations of the JET API spec. See
| notes bottom of this page: https://docs.microsoft.com/en-
| us/windows/win32/extensible-st...
|
| I say "we" because I am one of the developers who has worked on
| the engine for the last 15 years (around half of it's
| existence).
|
| Since I have worked on the engine, it has had a limit of
| slightly less than 2^31 database pages, which since it has a
| variable page size, has historically been 4 to 64 TBs.
|
| Even before the 2nd rewrite in the 90s, I believe it had a 16
| GB limit, so I really think the 2 GB is JET Red.
|
| Brett Shirley [MSFT]
| dlod wrote:
| Yes, Exchange 4.0 databases had a 16Gb limit and it was
| recognized then that this was insufficient. The next release
| increased that limit dramatically.
| bombcar wrote:
| This explains ONE of the causes of corruption of PST files back
| in the day.
| dlod wrote:
| PST never used this storage engine. It used NDB.
| ajb wrote:
| "... in order to stay on the safe side with the very first
| release of the source code, we have temporarily removed all
| comments"
|
| This sounds weird, wonder what they're worried about here:
| Personal Information? Profanity? Or just airing their dirty
| washing in public.
| based2 wrote:
| https://en.wikipedia.org/wiki/Extensible_Storage_Engine
|
| https://en.wikipedia.org/wiki/Microsoft_Jet_Database_Engine
| karmakaze wrote:
| I thought this was the Jet of MS Access but they call that JET
| Red.
|
| From the ESE wiki page:
|
| > For JET Red storage engine of Microsoft Access, see Microsoft
| Jet Database Engine.
|
| > Extensible Storage Engine (ESE), also known as JET Blue, is
| an ISAM (indexed sequential access method) data storage
| technology from Microsoft. ESE is the core of Microsoft
| Exchange Server, Active Directory, and Windows Search. It's
| also used by a number of Windows components including Windows
| Update client and Help and Support Center. Its purpose is to
| allow applications to store and retrieve data via indexed and
| sequential access.
|
| > ESE provides transacted data update and retrieval. A crash
| recovery mechanism is provided so that data consistency is
| maintained even in the event of a system crash. Transactions in
| ESE are highly concurrent making ESE suitable for server
| applications. ESE caches data intelligently to ensure high
| performance access to data. In addition, ESE is lightweight
| making it suitable for auxiliary applications.
|
| > The ESE Runtime (ESENT.DLL) has shipped in every Windows
| release since Windows 2000 [...]
|
| I was wondering how that might fare with the SQL Server edition
| meant to be bundled with apps. Besides being non-SQL seems it
| could be lighter keeping transactions and concurrency.
| sradman wrote:
| This Windows only embedded engine has long been part of
| Microsoft Exchange and Active Directory servers. The API was
| public but not well documented. Berkeley DB was a quality
| cross-platform alternative but I suspect that some permutation
| of RocksDB now dominates this use case.
|
| It should be noted that local SQL Server using fast IPC
| communication is quite good and rumors of Microsoft Exchange
| replacing ESE with SQL Server have popped up many times over
| the years.
| e12e wrote:
| Oh, so this is source code for "JET blue" under MIT license. For
| now without comments, tests and build scripts - but that's
| comming.
|
| Looks a bit like Microsoft's c++ toolkit that serves a similar
| use as lmdb, I guess? (although jet seems to do more than "just"
| db).
|
| https://symas.com/lmdb/
| dmw_ng wrote:
| Looks like it's slightly more abstract than LMDB, in LMDB
| you're given sub-databases on which to implement your own
| index, this sounds like it handle some of the boilerplate as
| part of the framework. The mention of support for schemaless in
| the README suggests it might possibly also define row
| structures and stuff like that
| dlod wrote:
| This code base in indeed old. It began life in March of 1989 at
| Microsoft and was originally written in C.
| tomc1985 wrote:
| Cool. I remember picking up issues of Visual Basic Programmer's
| Journal as a kid and seeing mentions of JET with no clue what it
| was.
|
| Neat that it is still around. A shining example of code being
| eternal!
| pjmlp wrote:
| So they are open sourcing Jet.
| jen20 wrote:
| Specifically Blue, not the Red variant used in Access.
| pjmlp wrote:
| Ah, I thought they were the same actually.
| dlod wrote:
| No, never the same but they shared a relational query
| engine called QJET.
| ThePadawan wrote:
| I know this as ESENT (since this is apparently the name of the
| .dll). Good to know it's still around!
|
| I saw this used as the "we really need to plug this performance
| hole" caching solution on a web server.
|
| Without knowing too much about how well it did, I have to assume
| it does especially well in combination with IIS?
|
| Would be interesting to find out how tightly coupled these
| benefits, if any, are.
| jeremycarter wrote:
| I believe the original RavenDB was based on ESE (ESENT).
| temac wrote:
| It is curious they even released that in this state where all the
| comments are stripped. I would not really consider it free
| software in this current state (at least this is probably not GPL
| compatible, because this is not the "source code" by the GPL
| definition -- this is not the "preferred form of the work for
| making modifications to it")
|
| But they plan to re-release with "cleaned up" comments. It will
| be way more interesting then, and with no ambiguity about its
| free software compatibility status.
|
| Edit: to be clear I'm not really complaining, it is more that I'm
| eager to see the real thing, and also wanted to remind people
| what "source code" is by the definition of an important licence
| for free software. Here this is specified as "MIT License", which
| is often considered compatible with GNU GPL v2 or v3: but be
| warry that it is doubtful this is compatible with the GPL in this
| state, you'll have to wait a little if you want to do an
| integration in this direction.
| [deleted]
| cptskippy wrote:
| Many times the comments are the reason code isn't open sourced.
| Companies want to audit them to ensure there's nothing damaging
| before releasing. The same goes for variable names.
|
| This is a whole lot better than nothing.
| justin66 wrote:
| TIL the software responsible for almost exactly 10% of the
| utterly inexplicable Application warnings in Event Viewer on my
| Windows 10 PC is considered by someone to be "server-grade."
| 2bitencryption wrote:
| I love perusing different MS open-source codebases, just to see
| all the different conventions, styles, and practices used by
| different teams across the company.
|
| This repo in particular sure shows its age, because just clicking
| around, this code is totally nonsensical to me, haha. I couldn't
| even make a guess as to what it's doing.
|
| https://github.com/microsoft/Extensible-Storage-Engine/blob/...
| jeffreygoesto wrote:
| Isn't that obvious?
|
| https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
| brettsh wrote:
| This is good feedback for the project, thank you. I have lived
| with the naming convention, so long that it is second nature
| ... I didn't even think of documenting it ... I will write up
| something on the naming convention of ESE on the gibhub
| project's wiki soon!
|
| Brett Shirley [MSFT] (yes, the Brett Shirley contributor to the
| project ;-)
| quasse wrote:
| As well as stripping comments, did they also strip and replace
| variable names? This code reminds me of trying to grok
| autogenerated Java GUI code in CS101.
| vbezhenar wrote:
| BOOL fVisible; INT cbKey = 0; BOOKMARK
| bmSearch; ULONG cbReq;
|
| I don't think so.
| Avery3R wrote:
| Nah, it just looks like microsoft's usual hungarian notation
| https://docs.microsoft.com/en-us/windows/win32/stg/coding-
| st...
| dlod wrote:
| Yes, Hungarian was fresh in 1988 so naturally heavy use in
| this code base.
| mypalmike wrote:
| I haven't seen Hungarian notation in so long. It's so
| painful to look at. And yet back in the day I too was
| convinced it was a good idea.
___________________________________________________________________
(page generated 2021-01-30 23:01 UTC)