Post A5adZirLyf6m0PkENM by PawelK@noagendasocial.com
(DIR) More posts by PawelK@noagendasocial.com
(DIR) Post #A5adZfZsBCLJpKXjDE by agates@podcastindex.social
2021-03-26T01:12:10Z
0 likes, 0 repeats
As part of alternateEnclosure testing, I'm trying to figure out the best way to represent optional metadata based on the format type of the item.In line with my "podcast:format" proposal (thanks @theDanielJLewis for the name), this is primarily to minimize guesswork for the client.Question to the community: How might we approach adding format-contextual metadata? See some attach screenshots for some examples of audio/video/scientificPaper -- just some suggestions of what's possible.
(DIR) Post #A5adZg1AXiB3BzJWl7 by PawelK@noagendasocial.com
2021-03-26T01:21:38Z
0 likes, 0 repeats
@agatesAnother approach is having hard well defined schema of complete max extent keys and values etc.ie hard schema.Takes more time in design, allows more optimal storage than keysvalues. Easier developing processing of data.@theDanielJLewis
(DIR) Post #A5adZgSosuIMZkFbrE by PawelK@noagendasocial.com
2021-03-26T01:24:28Z
0 likes, 0 repeats
@agates @theDanielJLewisAlso then schema is less fluid and changing less often and you dont have to update apps continuously.
(DIR) Post #A5adZgqvRHZrmVWrQm by agates@podcastindex.social
2021-03-26T01:34:34Z
0 likes, 0 repeats
@PawelK Yeah, whatever we do we need at least a suggested standard, if only because some developers probably don't even know what "schema" means (I wish I was joking).@theDanielJLewis
(DIR) Post #A5adZhH9rkYr5rnoJs by PawelK@noagendasocial.com
2021-03-26T01:38:24Z
0 likes, 0 repeats
@agatesCan you define roughly objects and properties you need already plus evolution of tags schema?I would like to have a closer look at this thinggy.@theDanielJLewis
(DIR) Post #A5adZhhOIDXqPE4lCy by agates@podcastindex.social
2021-03-26T01:40:02Z
0 likes, 1 repeats
@PawelK It's part of the Podcasting 2.0 effort you've likely heard Adam talk about on No Agenda.I think the whole point is evolution :)Documentation here:https://github.com/Podcastindex-org/podcast-namespace
(DIR) Post #A5adZi5UqapLbzM0mW by PawelK@noagendasocial.com
2021-03-26T01:41:03Z
0 likes, 0 repeats
@agatesIll have a peek. Maybe something pops to my mind.
(DIR) Post #A5adZiQ3c9H2dkyQpU by agates@podcastindex.social
2021-03-26T01:42:45Z
0 likes, 0 repeats
@PawelK Ultimately it comes down to what a client would implement... if they wouldn't bother with something, I wont bother... even if I disagree with where I see it headed 5 years in the future.
(DIR) Post #A5adZirLyf6m0PkENM by PawelK@noagendasocial.com
2021-03-26T02:26:56Z
0 likes, 0 repeats
@agatesOk read it i seem to understand the problem ok. Questiona are how many input formats will there be and how fast to add new ones and on whose side is conversion to your common format. If you can affird freezing schema and new firmats are added rarely schema is ok. If you need to have max flexibility and supperfast adding new input firmats go for keyvalues. Though in long run from osm keyvalues are stored in db in inefficient way and processing keyvals is problematic there.
(DIR) Post #A5adagE5FnLa7zpiUq by PawelK@noagendasocial.com
2021-03-26T02:29:04Z
0 likes, 0 repeats
@agatesProblem with keyvals in osm is there is lots of misnamed/invalid keys and vals and you need checkers and autocorrect process etc. Generally keyvals seem flexible but messy.
(DIR) Post #A5adeiCdqWjtPv55xQ by PawelK@noagendasocial.com
2021-03-26T02:27:49Z
0 likes, 0 repeats
@agatesI generally prefer schema whenever possible. Keyvals should be fallback or if schema is impossible.
(DIR) Post #A5adkC6i5e15byv9uq by PawelK@noagendasocial.com
2021-03-26T01:19:22Z
0 likes, 0 repeats
@agatesNot sure it helps but possibly openstreetmap had similar problem.They store 1..n key=tag pairs on each object. Theres no fixed schema of allowed keys and values and their interpretation is fluid. Some keys and or keys+values pairs are interpreted in some sense and its documented in wiki what keys what values etc.Its one approach. Flexible storage byt hard processing.@theDanielJLewis
(DIR) Post #A5afPMA3iL5QcXg97g by PawelK@noagendasocial.com
2021-03-26T01:40:22Z
0 likes, 0 repeats
@agatesSimilar problem to one in osm honestly. They had xml you have json. Treating both as base of db schema is eternal hell.@theDanielJLewis
(DIR) Post #A5g5vheVChvsTEjSJU by TurdFerguson@podcastindex.social
2021-03-28T19:08:52Z
0 likes, 1 repeats
@PawelK @agates in the ingestion process are you building in a way to log the errors and send notifications back to the submitter? Obviously, they can ignore them like most users do but they at least get a notification of the error and if a value is not critical the process will succeed where critical errors will fail?
(DIR) Post #A5g6EWPHEsAH5xaH6O by PawelK@noagendasocial.com
2021-03-28T19:12:22Z
0 likes, 0 repeats
@TurdFergusonIn their process no. There are no autocheckers notification to user or decline on add keyval afaik.Maybe for critical errors they decline. Not sure here 100% honestly.@agates
(DIR) Post #A5g6PIFoCNVMO2I95M by PawelK@noagendasocial.com
2021-03-28T19:14:19Z
0 likes, 0 repeats
@TurdFergusonGenerally their users fancy inventing new keys and vals. Schema control development evolution process is not very transparent. Roughly speaking community and consensus based but nonideal.@agates
(DIR) Post #A5g6jZdhGoV6688TKq by TurdFerguson@podcastindex.social
2021-03-28T19:17:58Z
0 likes, 1 repeats
@PawelK @agates hmm, I put out a post asking for the details of the data structure maybe I will take a look and see if they is anything we can do. I am also interested in how agnostic the dB is in case it has to move to a cheaper or newer product. Flexibility and portability important
(DIR) Post #A5g77MLGMRAro5r4sr by PawelK@noagendasocial.com
2021-03-28T19:22:16Z
0 likes, 0 repeats
@TurdFergusonIm curious and interested in contributing random bits if they have any value.@agates
(DIR) Post #A5g7bIgMX2r39008Rs by TurdFerguson@podcastindex.social
2021-03-28T19:27:40Z
0 likes, 1 repeats
@PawelK @agates I'm in the same boat. It sounds like they have it under control but wanted to look around just the same
(DIR) Post #A5g7g58GL13AZFKb20 by PawelK@noagendasocial.com
2021-03-28T19:28:34Z
0 likes, 0 repeats
@TurdFergusonHeh yep. #me2@agates
(DIR) Post #A5g7hSO9MDGQz8Pz60 by TurdFerguson@podcastindex.social
2021-03-28T19:28:48Z
0 likes, 1 repeats
@PawelK @agates yikes, that is no good at all. You can have standards and still be flexible.
(DIR) Post #A5g7nPTD7XZ7fgcjiK by PawelK@noagendasocial.com
2021-03-28T19:29:52Z
0 likes, 0 repeats
@TurdFergusonId guess so. They know better.@agates
(DIR) Post #A5g8V7LnqjsIDU05PU by TurdFerguson@podcastindex.social
2021-03-28T19:33:02Z
0 likes, 0 repeats
@PawelK @agates I am working on unwinding several databases now that all had there own undocumented development standards that changed depending on who was implementing. Unfortunately, due to this we are not able to lift and shift to a new platform in organized sprints.
(DIR) Post #A5g8V7iqT4J3MwmUKG by PawelK@noagendasocial.com
2021-03-28T19:37:46Z
0 likes, 0 repeats
@TurdFergusonImho dbs generally are one of messier areas of it world. No standards no automation tools for schema evolution etc etc plus damned may be sql for renowned bobby tables and so on.Also if going for keyvals there might be good nonrelational storages avail. Forgot the names but either they are open source or done by google or both.@agates
(DIR) Post #A5g8azgHHKXEAONZrM by agates@podcastindex.social
2021-03-28T19:38:16Z
0 likes, 0 repeats
@TurdFergusonTake that and multiply it by everyone and their dog who wants to create a website/podcast. RSS was created to be a simple and dumb standard to represent distributed content -- things like readability over usability for programs.It worked, but now we have a schemaless mess where everyone sort of follows a standard but often don't even know any better. That's kind of what we have to live with... suggestions that are not difficult to implement.@PawelK
(DIR) Post #A5g94ufMUJ0jIjDXTE by PawelK@noagendasocial.com
2021-03-28T19:44:14Z
0 likes, 0 repeats
@agatesYep good analysis of some sources of complexity explision we are observing.Everyone reinwenting a wheel in constantly evolving and replicating prog languages. 1mln frameworks in eachl of em, everything moving towards worst ones plus complete mess between tiers and components ie vms simulating cpus in js injecting jity assembly into them etc. Everything moving towards html http json xml node.js etc. Mess mess mess.Im bit depressed when i observe it.@TurdFerguson
(DIR) Post #A5g979NQlSG7VMtgJs by TurdFerguson@podcastindex.social
2021-03-28T19:38:21Z
0 likes, 1 repeats
@PawelK @agates to be clear, I am talking about standards for the database structure and interaction. If done correctly you can have multiple different ways to interact that are simple to complex. Possibly backwards compatible to an extent.
(DIR) Post #A5g9I44iogYUFOLneK by PawelK@noagendasocial.com
2021-03-28T19:46:37Z
0 likes, 1 repeats
@TurdFergusonIt needs architecting design docs idled protocols/apis care when implementing but oh oh we are late already. Lets deliver anything roughly workable. We will implement architecture later. Hehehe.@agates
(DIR) Post #A5g9abOHglv2mnL9A8 by PawelK@noagendasocial.com
2021-03-28T19:49:59Z
0 likes, 1 repeats
@agatesAww and for webapps its complete mess ruby schruby jsp php fb'shack gigglesgolang for web complangs etc etc. Webservers implemented in node.js. kill me if that bitch is not gonna explode visually beautifully one day.@TurdFerguson
(DIR) Post #A5g9n11uBnXRmMn9BQ by TurdFerguson@podcastindex.social
2021-03-28T19:52:11Z
0 likes, 1 repeats
@PawelK @agates think about it this way the scene is very young so there is so much opportunity to improve.
(DIR) Post #A5gA60FPK0gwr5A38a by agates@podcastindex.social
2021-03-28T19:55:38Z
0 likes, 1 repeats
@PawelKThere is SO much software with SO LITTLE eyes on it now days that it's amazing anything still runs. I think we should be thankful for open source software on that font... and open standards.I truly believe there aren't enough developers to keep up at this rate, and the industry just wants more and more business software using existing standards.The fact is we need need to make it easier for machine-machine communication, that's my life goal.(and I don't mean AI)@TurdFerguson
(DIR) Post #A5gA7Kl8XqnMlEdXyi by PawelK@noagendasocial.com
2021-03-28T19:55:54Z
0 likes, 0 repeats
@TurdFergusonOf course it is my hope. I for example envy llvm+clang for architecture. Chris Lattner is my hero if not demigod. If one would like to cleanup gcc and/or others, they should observe learn from and follow his architectural taste.@agates
(DIR) Post #A5gAFABpj3KQITrDQO by TurdFerguson@podcastindex.social
2021-03-28T19:57:18Z
0 likes, 0 repeats
@adam and @ Brian in Alabama how can I get some details around the database set up and ingestion process? I would love to take a peek to see how it is working.
(DIR) Post #A5gAMisZ9NaVuXRx0S by PawelK@noagendasocial.com
2021-03-28T19:58:40Z
0 likes, 0 repeats
@agatesYep 100%. This plus for me automating the process where possible. See flatbuffers/captnprito/corba idls and genning marshalling/unmarshalling code from it etc.@TurdFerguson
(DIR) Post #A5gAWmXCPlGa8yFhKq by TurdFerguson@podcastindex.social
2021-03-28T20:00:28Z
0 likes, 1 repeats
@PawelK @agates ha, I'm only a database dude. That is my wheelhouse
(DIR) Post #A5gAfbNtEvq0AV6UGe by PawelK@noagendasocial.com
2021-03-28T20:02:05Z
0 likes, 0 repeats
@agatesPlus on shortage of engineers explosion of langs frameworks implies explosion in projects and if we keep on this path earth wont be able to supply enough engineers due to physical constraints even if every fucking mosquito became a software devel.@TurdFerguson
(DIR) Post #A5gAmHQLqlnYI1RIa8 by PawelK@noagendasocial.com
2021-03-28T20:03:17Z
0 likes, 0 repeats
@TurdFergusonWe need everyone and everyone counts.I love dgraph project and graphql from your area.@agates