Big Arse Messages In Azure Storage Queues

We use storage queues for much communications between micro services in Azure. I ran into an issue, as in one instance messages were too large to fit into one queue message. Service Bus will let you have very large messages, but storage queues will not.

One co-worker suggested breaking down the large message into many smaller ones and send those, not a bad idea. I came across several posts that suggested combining blob storage and queues to send large amounts of data. Basically save message in a blob and send the blob’s guid as the message. I liked this better, and implemented a blob message producer. I think the idea has been around for awhile, but I got to say I like it, if you can’t use service bus that is.

Codemash 2018

A big shout out to my employer, Descartes, for allowing me to attend Codemash this year.

It has been a great experience so far.  What’s been interesting is how much the conference is ‘blockchain’ focused.  I doubt there was even one session on it last year, but I wasn’t here so can’t confirm that.  I had a nascent understanding of blockchain, but definitely have a much firmer grasp now.

I tried to focus my attendance on sessions that would assist me in my current work.  Since I work on web applications and services, micro and other, all on .net platforms I tried to focus on those.   I did sneak in a blockchain course or two as well.

Here’s a list of my notes from each class I attended:

Domain Driven Design: The Good Parts

Presenter Jimmy Bogard

  • Part 1
    • The Majestic Monolith
      • A super large system that does everything
      • Their first Domain Driven Project
        • Texas Juvenile Justice Department
        • The state building a system for each county to use
        • First agile project for Texas government agency
        • Contract was per sprint
          • In hindsight decided a bad idea
        • First DDD project for any Texas government agency
    • Project Goals
      • Increase Knowledge Sharing of data in county relevant to a particular case
      • Comprehensive system covering all aspects of juvenile process
      • Complete picture of Juvenile so can make good decisions about how to handle case
      • Very idealistic
      • Legal process in the state of Texas very complicated and part of system.
    • Since agile started making User Stories and Personas
      • Had 72 personas, that is a lot
        • Color coded by Agency
        • In hindsight, so many personas indicated perhaps should not be one big system
        • Different bounded contexts, perhaps by agency?
        • Lesson 1: Bonded Contexts are a real thing
      • Person table
        • Ambiguous Modeling problem.  One size fits all table
        • Lesson 2: Ambiguous modeling means should split system
      • Competing Priorities
        • Different agencies felt different things should be priority
        • Lesson 3: Core Domain needs consensus
      • Everyone agreed on the word ‘offense’
        • Although it had a different context and meaning for each agency
        • Lesson 4: ubiquitous language needs consensus
      • Huge interfaces and classes
        • Lesson:5 structural components are least important thing to focus on
      • Huge application, did ship and still being worked on, but one massive MVC system
        • Many compromises
        • Ended up being a big ball of mud, but it works.  Learned many lessons
  • Part 2:  The Do-over
    • Same client, but the adult justice system
    • Client didn’t want to build one big system again
      • Too costly and risky
      • Build app for one agency at a time
    • Applied lessons from first system that went somewhat wrong
      • Lesson 1: Cohesiveness brings clarity and deeper insight
        • Built around bounded contexts
          • System just for Prosecution piece
          • Built around the ‘case’ as that is most important thing to prosecution
      • Lesson 2: Flexible in places, rigid in others
        • They had a user/role/permissions mapping instead of hard coding roles to permissions
      • Lesson 3: Sometimes the model is a step up from their model
        • Built simple work flow
        • Work flow driven by queues
          • This replaced searching by case number, person and comments
      • Lesson 4:  Don’t blindly follow patterns
        • Code Changes
          • Collapse Layers
          • Had system model what it actually did
            • Get data to show
            • Update data
            • Controllers got thin, since just did gets and posts
              • Mediators took work from controllers
              • Mediators replaced repository
        • Organized project by Feature instead of by kind
          • Kept all pieces of feature together

 

Codemash 2018 – Event-Driven UX in the Real World with Angular and Socket.io

Presenter : Michael Meadows

  • Angular is now on version 5.2!
    • Angular plus websockets, plus reactive js extensions make for event driven ui, even when events on server.
    • He likes Angular and typescript, doesn’t care about reactjs
      • Angular works really well with reactive extensions (observables)
  • Why socket.io
    • small
    • stable
  • UX Patterns/Anti-Patterns
    • Anti-Patterns
      • Requeset/Response
        • Users don’t want to wait minutes for UI to update when waiting for response from long running process
      • Polling to know if something changes
        • Can crush your server if have a lot of usage
      • Save Button?
        • Pattern because used to update a whole file when posting
      • Optimistic Concurrency
        • First one to save wins
    • Good Patterns
      • Notifications
        • Broadcast changes from the server
        • What is happening on the server in socket.io
          • Three different ways to send events
            • emit
            • get users socket and emil
            • goes to just one client or to all sockets
          • Broadcast
      • Data Streaming
        • Event streaming
        • A client can add an item and tell web socket to emit the item
          • Server then emits the item to all other sockets
          • Bubbling event from client to server and back to client
          • Namespaces allow the limiting of what clients get emitted events when client says to emit an event
          • Room versus namespace
            • Chat may be the namespace the room is the actual chat we are in.
      • Record Locks
        • Can emit events from server when a record is locked and unlocked allowing clients to disable updating when record is locked and to know to update when lock released and to allow client to update again.
      • Soft Locks
      • CQRS
        • Command Query Response Segragation
        • Treat transactions and views separately

Codemash 2018 – Introduction to React + Redux + Redux Observables

Presenter: Jason Farrell

  • React is a library, not a Framework
    • Can use with Angular
  • Concept is “Render Always”
    • Virtual Dom pattern
      • Real Dom only updates with delta react detects in Virtual Dom
      • All rendering takes place against virtual dom
      • Render is called frequently so virtual dom forces constant rendering
      • Real dom changes are batched so real dom is only updated when necessary
    • State
      • Prop
        • Readonly
        • Passed in to component as attributes
      • State
        • Data help by the component internally
        • Never update state directly
          • Use SetState
            • Reason react pushes determinism
            • State is never updated, only replaced
              • Immutability
            • Never read child state from parent
              • To do this pass in callback to child as property from parent
  • Component Driven
    • Section of page is a component
      • Easier to manage
      • Easier to reusue
  • Redux
    • Manages state
      • Reducers listen for all actions, act on ones it cares about does nothing with ones it doesn’t
        • Each reducer manages a slice of state
        • Reducers are commonly if..else or case statements
        • type has to be a property of object passed to dispatcher as reducers use it to switch on
      • Container
        • Connected component
        • Container wraps component
          • import container in component
          • allows injections of props from container
        • A method called ‘connect’
          • short cut to map to dispatch and actions
          • mapStateToProps
            • Maps properties to values in the store that you want your property to change with
          • mapDispatchToProps
            • Maps actions that should be sent to store that your component kicks off
      • Store should never know about a ‘selected’ something as that indicates improper coupling to the UI
      • Top level components should rely on store for maintaining and passing state.
        • Low level components should use props to communicate with parents
        • Should not use store
      • Redux has one store, where flux has many
  • Briefly about RxJS – Observables
    • Observe and react to changes in a stream
    • Epic is an “Observable aware Reducer”
    • Epics go out and get the data
      • upon completion they will raise an event reducer will act on
      • state only changes when data is returned
    • Epics must return an observable
    • Written by netflix
    • applyMiddleware must be imported to get Epics
      • Done in his configurestore method that is called in index.js