Big Arse Messages In Azure Storage Queues

We use storage queues for much communications between micro services in Azure. I ran into an issue, as in one instance messages were too large to fit into one queue message. Service Bus will let you have very large messages, but storage queues will not.

One co-worker suggested breaking down the large message into many smaller ones and send those, not a bad idea. I came across several posts that suggested combining blob storage and queues to send large amounts of data. Basically save message in a blob and send the blob’s guid as the message. I liked this better, and implemented a blob message producer. I think the idea has been around for awhile, but I got to say I like it, if you can’t use service bus that is.

Agile Is About Velocity, Not Standups

It seems to me that any good process should provide assistance in four basic areas.

  • Predictability
  • Quality
  • Throughput
  • Morale

Predictability is important because you want to be able to tell people approximately how long something will take.  Quality is important for obvious reasons.  Throughput, the amount of work  you get done, should be positively impacted, and morale should go up if your process is a competent one.

Agile is handy for the above if you can create a meaningful velocity because it serves all four areas.  If you can effectively break down and point stories and have a meaningful velocity, you can tell people more accurate information about when you will be done.  If you effectively break down and point stories and have a meaningful velocity you can measure throughput and know if changes you make increase or decrease your throughput.  If you effectively break down and point stories with the input of test and business stakeholders quality should theoretically improve, as well as predictability as stakeholders are more likely to get what they are asking for.  If you effectively break down and point stories, and involve all team members as equals, morale is likely to be high.

Stand up meetings are nice, but that does not make for effective Agile.  What makes for effective agile is tracking and maintaining an effective velocity.  To do so means you are effectively breaking down work into stories that testers and business agree are of value and are done.  Agile is really organizing yourself so you can create and track a meaningful velocity.  By doing this good things collaterally happen.    If you have stand up meetings, and scrum masters and task boards and no velocity you are doing something, but it isn’t Agile as far as I’m concerned.  Have only the meetings and artifacts required to track and maintain a meaningful velocity and the rest will take care of itself.

When a group says they embrace Agile, ask them what their team’s velocities are and how long they’ve tracked them.  If blank stares are the response, then run.

Automating Test Automation

We have finally started automating our functional testing. We have a large .net based web application and after much time I have convinced the powers that be that we should be creating selenium tests when we test new functionality so that we can retest that functionality later with the push of a button.

So first had to overcome creating same state for tests when they began, then had to overcome functionality common tasks. And now we are trying to automate the automation and have our selenium tests run on our build server. I’m all for it, but it is not easy.

Having issues as only headless browser that seems to work in a windows environment is PhantomJS. This would be ok except some of our tests send files down by changing content-type header, and PhantomJS doesn’t yet support this.

Trying to see if we can run Chrome headless, or setup a server somewhere to run tests so doesn’t matter if headless or not. Will update where we get to.

Codemash 2016 Notes: Convention Over Configuration: Queueing is Easy

Convention Over Configuration: Queueing is Easy

Presenter Matthew Groves

  • Queuing is intimidating
  • Quick review of queueing
    • processing that isn’t web or db can be pushed to another server and not impact ui or db server or make users wait for processing finish(async)
    • Introduce a queue instead of managing async communication on our won
    • We take request from user, tell them we will get back to them stick request on queue
      • can scale processors taking work off the queue when needed
  • RabbitMQ on .NET demo
    • reviewed basic of running rabbitmq on .net
    • rabbit messages are byte arrasy so have to serialize payload to byte array
    • have to acknowledge receipt, basicAck, or message will stay in queue.
  • Queuing patterns with EasyNetQ
    • .net tool only
    • Tool that sits on top of rabbitMQ
    • provides common queuing patters right out of box eliminating need to do deep configuration and serialization and deserialization of messages.
      • can think of messages as objects now, no need to get to byte array on your own
      • best of .net on both client and receiver
    • common patterns
      • Send and Receive
        • send message to a receiver, duh.
      • Publish and subscribe model
        • publisher puts message on queue doesn’t care who is listening
        • subscribers listen to what interests them
      • Request and Response
        • put request on request queue
        • put response on response queue

Codemash 2016 notes: ASP.NET 5: How to Get Your Cheese Back

ASP.NET 5: How to Get Your Cheese Back

Presenter Tugberk Ugurlu

Surprised by how much has been changed in ASP.net 5.  I think the changes are good, but concerned about lack of backward compatibility.  MVC is now free of system.web so can be deployed sans IIS.

  • ASP.net 5 is entire rewrite
  • All files are part of project have to exclude what don’t want
  • Unified dependencies
    • can set dependency by framework
  • Project file is not .json
  • Configure method in startup, similar to owin and katana
  • New concepts
    • Command line tools
      • dnvm: dot net version manager
        • manages the versions of .net on your machine
        • Can assign different framework version to different processes running at same time
      • dnx: dot net execution environment
        • dnx run will run your application
      • dnu: dot net utility
    • Command-line first development ?
  • Modular, Composable HTTP Pipelie
    • pipeline is entirely empty to start have to add everything
      • good since don’t have to pay the price for things you don’t use.
      • in startup configure method you add to the pipepline what you want
  • Dependency injection all the way down
    • built in container get swap out with your favorite
    • ConfigureServices method in stratup used to config DI
    • ConfigureOptions<T>, syntax for configuring, like identity
  • Config
    • in startup you tell app where and what kind of config files exist
    • configurationmanager is all gone, where it was used you need to change
  • Unified mvc and webapi
    •  removed mvc dependency on system.web
  • Code editor agnostic
  • Cross platform now
    • can run windows,linux, osX
    • kestrel web server is supported
    • Can be dockerized now