Ruby is awesome, almost as good as Python. I have to work up the motivation to learn Go and Rust. I hear those are the up and coming languages for all the new toys.

Python has been of recent intrest. Seems the only Tensorflow Keras impls.

Damn data nerds.

I have been working for quite a while on a unique problem space that is on platform that is C#, SQL, on AWS. a SAAS solution for managing intellectual property rights.

Python and R are the two defactor ML and AI languages so you will have to pick one.

Wish I had more time to experiment with other environments.

Can't say I miss writing C#.

Oh, that sounds like a fun problem. I'm just learning Azure. Have some experience with AWS, Google Cloud, and some other small providers.

Azure eh? any feedback?

I wishing banking and finance would have a big push to the cloud. It would be so much more secure and easier to manage than our on-prem stuff.

The offerings for each are nearly identical, VMs, containers, virtual networks, big data, AI and ML, relational and nosql databases, storage, hot failover, etc. It's just learning the details of whats were and the little unique issues.

*what is where

IP rights are defined by combinations of hierarchical data sets. A contract can be defined in a few hundred records, but contract processing requires comparing the cartesian sets of each in order to determine potential overlap or gaps. A single contract line item could expand to billions of rows. The resources required can be daunting.

If you are interested in playing with it Microsoft Learn walks you through the environment and lets you spin up environments for free. https://docs.microsoft.com/en-us/learn/

I change my mind, that doesn't sound like fun.

I'm quite adept at Azure 😄

I'm getting tired and must have misunderstood your comment. Again, same thing different skin. I'm honestly surprised by how similar all the platforms are. It's like they clone the same underlying architecture to some extent.

I heard rumors that they were all built on openstack but I'm not sure how true that is.

I was able to devise a way to use binary data arrays to represent the complex hierarchical relationships in a single row and discovered a math trick that lets me make those types of calculations with a fraction of the resources needed. Using traditional SQL techniques, a common query could take 1.5 to 2 hours to complete on an optimized server. I was able to get that same result in 6-8 seconds with 100% fidelity... It's not often I talk with people that will know what I am saying, so I hope you don't mind.

😄

We did a similar thing with calculations our bioinformatics department was doing on mouse DNA while I was in graduate school. Someone had written a python script to process all the data on a cluster of high powered machines and it tooks about 36 hours. We rewrote it to using CUDA and cut it down to about 5 minutes. Really just a whole bunch matrix math and graphic cards are optimized for that type of operation. I'm sure you can't share the details of algorithm but it sounds very interesting.

Yeah.... like I said it's basically a binary math trick. I figured out that operations like subtracting one set of sets from another set of sets could be done without needing to know anything about the underlying relationships. part of it is how you set up the arrays - defining each element as the full representation of the constituent parts. For example, a field representing territories would include a binary array where each bit represents a country (to keep it simple, it could easily be region, state, county , city). A contract line item for North America would have the bits corresponding to Canada, USA, and Mexico set to on and the rest off. The same is true for things like media, rights etc. Then there is a predictable pattern that is applied and the result is the product. I know it's vague, but you hopefully get the gist. I am able to use VARCHAR(8000) fields to represent up to 63000 items.

That's one of the things that is getting rusty for me in security is algorithms and data structures. Lots of assembly, shellcode, scripting, crypto, etc. Not as much optimization and engineering..

Sounds like a fancy bitmask. There are a lot of neat tricks you can do with bitmasks.

It's counterintuitive, because you can't index those fields and you can only process binary logic on a BIGINT field, but I figured out how to do it and it works like magic.

If I do say so myself... 😄

That is a very clever solution. I wouldn't have imagined that VARCHARs would allow you to do the types of operations you are describing. I've never used them that way.

I find that I'm always proud of myself when I use a bitmask, so I understand the feels.

I love bit masks. They are so efficient and elegant.

spoken like a true python dev

Regarding the tariffs... I think that people may not realize that the reason that companies are able to safely transport things to/from China because the US Navy actively polices the trans-oceanic shipping lanes. People are content to pay less for their cheaply made goods, but don't realize that they are paying for it down the road through deficit spending on the military. If companies were forced to pay their fair share for the cost of policing the shipping lanes, it might not be as cost effective to have things made in Asia.

It's funny... bitmasks are so freaking fast and use almost no disk/processing resources. What I give up on indexes, I make up a thousands times over.

bitmasks are almost like programming to bare metal, so it should be fast.

I get it. Bitmasks are awesome. So few processor cycles to do relatively complex operations.

That's a lot like the Walmart effect, where Walmart ends up be subsidized by entitlements programs and ends up making an unfair profit.

@linuxace, you just advanced to level 7!

I'm a free market guy but stuff like that needs to be remedied.

If you appreciate efficient programming, I came across a guy in Australia that does the coolest things in Assembler. Little utilities that a full functioning apps in hundreds of KB's... That's right K B's. It's pretty amazing. Check out http://sinnercomputing.com/

That site is getting bookmarked.

I mean, Walmart employees don't get paid enough and go on welfare and Walmart is essentially stealing tax dollars in a round about way.

It's totally the Walmart effect. They are able to get cheaply made goods from China - so we can pay $200 for a microwave that is going to fail as soon as the warranty ends. Then we buy another one. Of course, the sunk cost of ocean security was just added to the tab that we are all on the hook for.