<CharlieDigital/> Programming, Politics, and uhh…pineapples


The Russian Long Game

Posted by Charles Chen

If you only read one crazy Trump-Russia conspiracy theory, read mine ūüôā

Politicians and military leaders sound increasingly belligerent and defense doctrines more dangerous. Commentators and TV personalities are joining the bellicose chorus. It all looks as if the world is preparing for war.

Mikhail Gorbachev - Time Magazine, January 16, 2017

Right at this moment, the government of the United States is plotting how to start at least one war, but probably two.  The reason?  Oil.  Specifically, the value of oil and its importance to the Russian financial situation and geo-political power.

Global Oil Chokepoints - I've highlighted some key regions; keep in mind as you read on.

EU sources of energy imports.

All the pieces seem to align.

It seems like a priming of the conditions for oil assets to skyrocket. The problem is that there's more supply than there is demand right now so even if Russian sanctions are lifted, it does little to help the situation in Russia...unless the price of oil can be inflated somehow by disrupting supply or increasing demand. Increasing demand is hard; economies have to grow, the middle class has to be enriched, people have to buy things -- super complicated economics that will require the world's wealthy to give up some of that wealth. But disrupting supply.....

In contrast, the Obama administration strategically crushed the price of oil:

Obama viewed the price of oil as a strategic tool to weaken our enemies. Lower the price of oil and lower the influence of countries like Venezuela, Russia, and oil producing countries in the Middle East. When he took office, the price of oil was $102/barrel. When he left, it was in the mid-thirties. Those last four actions if evaluated by themselves, would seem to be typical liberal causes; but in the context of his administration's actions (or non-actions) on oil drilling, they were part of a coordinated effort to weaken the influence of oil prices on geo-politics. By crushing the price of oil, it would naturally slow down demand for "Drill baby, drill". What happened to ANWR, the topic du jour of 2008? Nothing because the glut of oil in the world destroyed the need for ANWR drilling; why drill in Alaska when you can drill in Oklahoma and the Dakotas at a cheaper price? Crushing the price of oil and adding sanctions to Russia weakened Russia tremendously from a geopolitical and financial standpoint.

Once this framework is established, a lot of odd actions by Trump start to make sense.  Consider the case of Ford, who manufactures their best selling vehicle -- not just their best selling vehicle, but the best selling vehicle in the US -- in the USA.  Why would Trump give them heat for moving production to Mexico?  Well, the production they were planning to move to Mexico was for their compact cars which have lower prices and lower margins than their pickups and SUVs.  What happens if you force that manufacturing back to the US?  Suddenly, the price of compact and midsize cars go up and the price of an SUV or a pickup becomes more attractive.  And guess what?  You'll consume more oil.

Putin is playing a brilliant long game. Weakening the EU, weakening the US, weakening institutions that the West has relied on like the UN and NATO. Destabilizing Syria. I think Ukraine was the moment that he knew he was going to win (they shot down a commercial jet liner for crying out loud!).  Ukraine was their test bed, their pilot for how to execute a mis-information and propaganda campaign both in Western media and on the ground with those supposed separatists.  Brexit followed and then the US. Hillary Clinton totally discredited...on what basis (hey Mr. Chaffetz, how about we restart those Bengahzi hearings until Hillary's in jail -- she's a criminal, right?!  Her crimes are surely not absolved because shes lost the election and I'd be quite upset if Trump pre-emptively pardoned her after all those hearings and no conviction)? EU is becoming more and more nationalistic and fracturing. Russian geo-political victory is now in full swing. Price of oil is up to $50/barrel today. A "terrorist attack" from "Iranian nationals" or "an Iranian supported cell"? $100/barrel crude. Boom.

With that context, this visa ban is a sleight of hand that the media has fallen for; it's a set up to topple the judiciary, the media, and the moderates - Democrats and Republicans alike.  We all know that it's not constitutional.  We all know that it will fail at the SCOTUS; no one in the Bannon cabinet believes that it will pass the test.  But that's not the point; the point is to use this to discredit the judiciary and the "liberals" who are right now protesting against this by letting an attack happen (again, why else do you need Bannon to control the flow of intelligence?) and then using that as pretext to start a war.  Remember the last time this happened?

Obama played the long game and played it well. The difference is that he played it at a policy level. The brilliance of Putin's Russia is that they know they cannot win at this level -- they don't have the allies to do so -- so they have chosen instead to turn the power (or should I say weakness) of Democracy on itself and use nationalism, racism, and ignorance to power their agenda. Had Brexit lost, had Hillary won -- world looks very different for the next 10, 20 years.  We're talking about coal and oil again when we should be talking about solar and wind.  We're talking about a $20+bn wall instead of an $20+bn invested in an upgraded electrical grid.

While Americans were arguing about abortions and guns and fake promises of American manufacturing jobs and calling a 30 year public servant a Wall Street profiteer and criminal....and then actually voting for a Wall Street profiteer and super corrupt candidate, Russia pulled off a brilliant coup to leverage the most powerful country on the face of the planet to ply their agenda.

It is likely that Bannon will have a key role in instigating or facilitating the events that will unfold; why else would one place an unqualified individual to lead a national security council but to allow manipulation of intelligence and keeping professionals and adults out of the fold.  He will have more power and leeway to control the narrative and shape the story; after all, he is a master story teller -- a craftsman of alternative facts -- not a foreign policy or intelligence wonk.  Not by a cosmic light year.

Chumps like Jason Chaffetz are so blinded by the short game -- small ball -- that they don't see their critical part in the bigger picture of global politics.  They are tools, pawns to be used and discarded in this game.  He won't survive the next election, but he's already served his agenda anyways and can otherwise be sacrificed.  He wasn't in on the game anyways, he was playing Chutes and Ladders while Putin was playing Chess.

So what can be done at this moment in time?  The only hope is that Congressional Republicans pursue the Russian connections with as much fervor as they did Hillary's supposed criminal acts in the attacks on the consulate in Benghazi.  Get Steve Bannon as far away from controlling the flow of intelligence as possible; he has no business in intel.  2018 is too late to wait for an opportunity to flip the House or the Senate; the administration and the Russians will be opportunists, using any incident to drum up the machinery of war.


An Alternate Meaning for FOCKED

Posted by Charles Chen

Eric Brechner came up with one of my favorite acronyms of all time in software development: FOCKED.


I want to add an alternate: Failure to Orchestrate Collective Knowledge Effectively for Delivery.

Successful delivery of software requires that different members of the team come together and understand the goals that have to be achieved and the priorities of those goals.

It's as simple as communicating to the team on a regular basis (no more than once a week, but maybe at least once a month):

  • where we are,
  • where do we want to go,
  • when do we have to get there,
  • how are we getting there,
  • who's driving

It can make the whole process of delivery of software much less stressful and maybe more successful simply by aligning all of the stakeholders periodically.

Hey, maybe you learned this in some fancy MBA class or something, but I'm starting to appreciate -- more and more -- that the real secret to successful delivery of software is driving the successful collaboration and communication of people and alignment of all pieces to a strategy, vision, or goal.  Having a bunch of smart, capable people doesn't help you much if no one knows what's going on.

Filed under: DevLife, Rants No Comments

Adventures in Poorly Written APIs

Posted by Charles Chen

I'm working with a library that I have been fighting with for the better part of three days now to try to get it to work.

The previous version of this library was actually very well written from an API perspective and well documented.  The new version?  Not so much.

But from this, I take away two lessons in API design.

Lesson 1: Don't Use Dictionaries in Public APIs -- ESPECIALLY for Required Data

The first mistake the API designers made in this case was exposing a "dictionary" in the public API.

This itself may not be all bad if the API is highly flexible and can accept many forms of requests, but if a public dictionary interface is required, then at least have the good judgement to provide a wrapper to populate those dictionaries.

But even the lack of usability is not as bad was having required data inputs passed to your service as dictionary entries.  In other words, the service call won't succeed without some sort of magical recipe of just the right metadata keys and values.  If it's a required piece of metadata, why not surface it as an actual parameter to the function?

Exposing dictionaries in public APIs is a terrible design idea which is as bad as APIs that require you to pass in XML strings (understood if it's a must, but have the courtesy to provide downstream callers an API to serialize objects into that XML and don't put required data into open ended inputs).  They should be used infrequently and preferably wrapped by an abstraction layer.

Lesson 2: Don't Require Your Caller to Keep Passing You Data You Handed Back

In this particular API, there is an initialization handshake with the server whereby large bits of system data are passed back to the caller.

For whatever reason, instead of simply handing a token back to the server, all (or part?? -- again, it's a magical incantation of sorts) of this metadata must be exchanged with the server to complete a request!

Why not provide a session key?  A caller token?  Something that the caller can use to represent this collection of system data instead of the full set of system data?

More to come, I'm sure of it, but this has been an interesting adventure in poorly written APIs.

Filed under: Dev, Rants No Comments

The Dichotomy of Change Control and Quality Software

Posted by Charles Chen

When processes attack!

When processes attack!
(Sybren A. St√ľvel via photopin cc)

It may seem counter-intuitive to the old guard that change control can actually negatively affect the quality of software, but when implemented in such a way that it dramatically increases the cost and effort of making software improvements, it leads a team down a path of mediocrity.

You see, good developers naturally want to fix things that they find wrong with the software.  Tweak it to be a little faster.  Make a button bigger so it's easier to see.  Change the alignment of some form labels.  There are endless ways that good engineers will reflect upon their work or the work of their peers and find better ways to deliver a superior user experience.  Good teams will also find ways to quickly fold user feedback into the UX design and improve the usability of the system.

Indeed, there are times when changes are drastic and require that proportionally sufficient effort is put into testing, but there are times when changes are small and well contained where running the same process for big fixes simply yields frustration and drives a team to ultimately "settle" for fear of waking the slumbering giant of change control.

Consider a doctor performing open heart surgery.  Of course, the protocol for sanitation and safety checks are going to be far more rigorous than for administering a flu shot.  But this makes sense because the risk is far, far greater with one than the other.  If a doctor's protocols for preparing for surgery were one-size-fits all and applied for administering a flu shot, I suspect the quality of care in the US would decline drastically while causing the cost to skyrocket!  Or consider the permits and documentation required if you want to add a walkout entrance to your basement.  Of course you will need engineering specs, signed permits, and so on because the risk is high.  No one would think of requiring certification from a structural engineer and permits from your municipality to hang a picture frame on your wall because it would be absurd!

Likewise, common sense, pragmatism, and a sensible balance is a requirement for any software quality process; a one-size-fits-all approach creates a wall of effort for even minor tweaks that simply means that the small fixes that can make a big difference in the usability and functionality of the system aren't made for fear of generating a ton of e-paperwork.  The Broken Windows Theory, popularized by Steven D. Levitt and Stephen J. Dubner in Freakonomics, explains this phenomenon:

In an anonymous, urban environment, with few or no other people around, social norms and monitoring are not clearly known. Individuals thus look for signals within the environment as to the social norms in the setting and the risk of getting caught violating those norms; one of those signals is the area's general appearance.

Under the broken windows theory, an ordered and clean environment ‚Äď one which is maintained ‚Äď sends the signal that the area is monitored and that criminal behavior will not be tolerated. Conversely, a disordered environment ‚Äď one which is not maintained (broken windows, graffiti, excessive litter) ‚Äď sends the signal that the area is not monitored and that one can engage in criminal behavior with little risk of detection.

In the education realm, the broken windows theory is used to promote order in classrooms and school cultures. The belief is that students are signaled by disorder or rule-breaking and that they, in turn, imitate the disorder.

When change is made expensive, niggling things here and there in the code and UX that don't get fixed simply accumulate over time and promotes discord and disorder.  When the processes discourages innovation and excellence, a team ends up simply mired in mediocrity with no real quality to show for it.  It's great that there's now a trail of paperwork long enough to circle the Earth for the changes that were made, but the real shame are the fixes, improvements, and ideas that weren't implemented because of the cost in paperwork.

Again, this is not to say that no quality processes or change control should exist, but that their application must be proportional to the risk and pragmatic about the priorities of the project.

One way to solve this problem is via rigorous automation for it enables teams to condense the processes into robotic automatons that take minutes, not hours to execute.  It enables teams to conceptualize, develop, and deliver more rapidly through continuous deployment.

Wired had a good article on how LinkedIn is able to build and release new features quickly:

LinkedIn is a Wall Street darling, its stock up more than threefold in two years on soaring revenue, spiking profits, and seven straight quarters beating bankers’ estimates. But LinkedIn’s success isn’t just about numbers: an impressive acceleration of LinkedIn’s product cycle and a corresponding revolution in how LinkedIn writes software is a huge component in the company’s winning streak.

Much of LinkedIn’s success can be traced to changes made by Kevin Scott, the senior vice president of engineering and longtime Google veteran lured to LinkedIn in Feb. 2011, just before the buttoned-down social network went public. It was Scott and his team of programmers who completely overhauled how LinkedIn develops and ships new updates to its website and apps, taking a system that required a full month to release new features and turning it into one that pushes out updates multiple times per day.

By creating the tools and process and training the team to operate under continuous deployment, LinkedIn was able to quickly bring concepts and ideas to life; it is because the cost of making changes and improvements to the software have become cheap that they can be made readily and without friction.

True software quality can never be achieved solely through heavy-handed processes (unless they are automated!); such processes are simply paper tigers that lead to the appearance of conformance but instead, create an impedance to creating better software.


Microsoft, SharePoint, and Enterprise Social

Posted by Charles Chen

Every once in a while, I'll talk to a colleague and the topic of SharePoint and "social" will come up.  I mean literally, every iteration of SharePoint adds some other "social" feature or another.  It always leads me shaking my head.

You see, Microsoft really talks a great game when it comes to social and SharePoint.  It's the next big thing!  It's all new!  Look at all this cool stuff that you can do!

But all you have to ask yourself is whether Microsoft has proven themselves in any aspect of social.  The answer is simple: no.  And social for SharePoint is no different.

There is a very fundamental reason why it's a non-starter: social for the enterprise is stupid.

"But doesn't NewsGator make millions of dollars?"

Sure, but social for the enterprise is still stupid.

"But didn't you hear?  They purchase Yammer!"

Yes, but social for the enterprise is still fundamentally broken.  Never forget: Google once offered Groupon $6 billion -- doesn't mean that it was ever a good idea.

At the core of why social and the enterprise is fail, it's actually pretty simple as it comes down to one main issue: Portability.

Of course, for most people, their social networks often include a nexus around co-workers and professional acquaintances.  The perfect case for social for the enterprise, right? Wrong!  Because LinkedIn already has that covered and, more importantly, a professional social network on LinkedIn is portable.  If I leave the company, I can still stay in touch with all of my contacts and connections at my previous companies via LinkedIn whereas my investment in a closed network is entirely thrown out the window the moment I leave a company.

A closed-system, non-portable social network is a useless one; it's a non-starter.

Aside from the major flaw of lack of portability, there are a few other shortcomings that come to mind.

The first is that, in a large company, I simply don't care about 99.5% of the people in the company; I'm only connected to these people because we happen to be employed by the same company, but that could change at any moment.  In a small company, I will already have everyone added in LinkedIn or Facebook.  What good is a closed-system social network when the people in that network would largely be people I already work with on a daily basis?

The second is that, in my opinion, what is of more value to teams are not social capabilities, but collaboration capabilities.  Social is like the antithesis of task-oriented collaboration capabilities.  Teams would get more value out of an investment collaboration capabilities than an investment in social capabilities.  What I need from my enterprise platform is not social, but tools to help me and my team get things done.  You want to know what other people are working on, but usually in the common context of a project or such so tools that are focused on communication and collaboration in the context of project workgroups seems much more useful and practical an investment to me.

The third is that many companies include a forum or discussion type system in their social initiatives.  I saw this at CSC where Jive was deployed.  But this kind of circles back to the closed-system problem: if I have a question or if I'm working on a hard technical problem, am I more likely to find an expert within my company?  Or am I more likely to find an expert within the global community?  This approach makes sense maybe within Microsoft or Google, where the concentration of highly specialized subject area experts is high.  But in a consulting company, for example, it is less likely to be of any value as the global community will include a greater breadth of subject matter experts with a more extensive depth of knowledge.

Microsoft and social -- especially in the enterprise -- is pretty much a pointless endeavor, in my opinion.

Filed under: Rants 2 Comments

Avoid West Elm

Posted by Charles Chen

It's hard to imagine how some e-tailers are still functioning like it's 1999 in the age of Overstock and Amazon.

I ordered an item from West Elm online on October 18th assuming that I'd have it in my possession within a week.

Nope!  A week later, and nary an email on the status of the order, which is still listed as "shipping soon" on their website.

Turns out on 11/09, I'll get a call to set up a delivery date....

I don't even get the privilege of setting up a delivery date until almost a full month later?  Are master craftsmen hand-crafting my headboard, right at this very moment?  Given the reviews of their quality (or lack thereof), I think not.

I'm not sure if their stock management system sucks or what but this seems pretty crazy (OK, I'm spoiled by Prime as well).  I think it's a bit silly that I had to contact a live person to get a status on my delivery.

Turns out that this is SOP for West Elm and would have avoided them from the start had I known.

Filed under: Rants No Comments

Oh, Visio, How You Rustle My Jimmies!

Posted by Charles Chen

I have been working on putting together an information architecture/file plan document which shows the layout of artifacts in our SharePoint environment and came across this nice little gem after spending a bit of time putting together my stencils:

As you can see, this is the multi-connector and it seems to work great.  But then I realized that it seems that the multi-connector only allows for 6 branches...

Amusing and frustrating because it makes no logical sense to me.

Filed under: Rants No Comments

Windows Azure “Introductory Special” — WTF?

Posted by Charles Chen

WTF Microsoft? Seriously?

This is what you call an "introductory special"?


This is fine and all, except in the months of January and February -- in fact, since I've signed up -- I've never used the caching service.  Heck, I've barely used Azure!  How can I possibly be billed for $46?  Yeah, I'm pretty much cancelling my sub after this; can't waste my time and energy every month double checking to see if I'm being ripped-off by Microsoft.  And on top of all that, I have to wait 4 hours for a support callback...

Now here is my theory: I am being billed $4/day just for having 128MB of cache allocated, whether I use it or not.  But this is beyond fucking stupid. It hurts adoption because it drives developers -- like me -- away from even signing up in the first place if I have to police by billing, even when I'm not using the service.  And seriously $46/mo. for having 128MB of web accessible memory available?  I get more memory allocated from my $7/mo. hosted shared server.  WTF?

I can accept the cost if there was some way to opt out or if there was some notice that I would be billed for being allocated the cache space or if there was some à la carte way to turn it off -- my mistake for not noticing in those cases.   But I was not given an option to opt out of cache, I was not notified that I would be billed regardless of actual usage, and as far as I can tell, there is no way to turn off cache.

This is some serious bullshit.

Filed under: Rants No Comments

Airline Boarding and How to Make It Suck Less

Posted by Charles Chen

Caught an unexpected segment in the news on airline boarding (ending is great):

I've mostly had (terrible) experiences with boarding Continental flights and I've often thought why the process is so screwed up.  I've come up with the following conclusions myself:

  1. Problem: Last time I flew out of John Wayne International, literally 3/4 of the flight was "Elite" status.  So when they did the pre-boarding, it was a huge throng of people boarding the plane at once, which negated the whole point of boarding the plane from the back first. There's something broken about the system here because it creates a weird paradox.  Being able to board the plane faster is a "benefit" of Elite status, but by using this system, it extends the overall boarding time, thereby actually making those with Elite status spend an overall longer period of time sitting on the plane.  Solution: Elite status is only used in consideration for upgrades and seat selection at check-in, not boarding order.
  2. Problem: Gate reps don't actually check the size of carry-ons.  In some cases, preemptively removing larger carry-ons and forcing gate-checks would dramatically reduce overall boarding time since the large bag causes an additive delay effect.  That is that it not only slows down the individual with the large bag, but also affects every passenger that boards after that passenger because a large bag taking up too much space forces "bin hunting".  Solution: Actually enforce the existing rules?  Seems too simple to be true.
  3. Problem: No one keeps their outerwear until the boarding process is over.  Nothing more frustrating than people stuffing their outerwear in the overhead before everyone has boarded.  This takes up valuable overhead bin space and, again, delays the boarding process for everyone that comes afterwards by forcing bin hunting. Solution: Add coat hooks on the seats and people will more likely just hook their outerwear on the back of the seat for easy access.
  4. Problem: Some people just don't give a damn what the gate rep calls -- they're just going to get onto the plane anyways. Solution: See below.
  5. Problem: Because of a combination of the above, it often causes folks to end up back-tracking and bin-hunting.  Sometimes, it also causes folks to preemptively place their carry-ons in forward rows, thinking that there is no space in the back rows, which only further exacerbates the situation. Solution: See below

The whole process is generally full of chaos, completely inefficient, and seems dated -- as if no one has stopped to think about these issues and how to solve them for decades.  So it's interesting to see this guy -- Dr. Steffen -- propose something new.  But it got me thinking that this guy's method still has an issue: it's not realistic for gate reps to call out this seating scheme (one row, one aisle at a time).  Yet this is what it demands as the process breaks down if you have passengers in row n-2 blocking passengers in row n.  Furthermore, the Steffen method makes an assumption about the nature of people: that they will obey order and think in the best interest of everybody.  When the gate rep calls the aisles for rows n, n-2, n-4, n-6, all of those folks are going to rush the counter at once in a single mass.

So I think an extension or more realistic approach to this method is necessary to get it to work: a new passenger queue system at the gate.  It should be -- like a plane -- one center aisle with several color-coded lines on each side of the aisle and just have people queue up in "bins" first.  It shouldn't match the rows in count, but should be at a 1:3 ratio per bin.  In essence, it's a modified block boarding pattern, but we break it down within the block so that, for example, window aisle passengers in the block are in bin "Purple", middle seat passengers are in bin "Blue", and aisle seat passengers are in bin "Green".

The gate rep would call bin "Purple 1" to board the window seat passengers in the rear of the plane.  Then call "Purple 2" to queue the passengers in the window seat in the middle of the plane, then call "Purple 3" to queue the passengers in the window seat near the front of the plane.  However, the real efficiency is in having the folks get into the bins first, before they are queued up.  The current Continental system results in 60 people all standing around, blocking the entrance of the queue when all that's needed is a bin system so they know where to line up.

This system acknowledges that the ideal boarding pattern cannot be achieved in real life and thus only attempts to gain efficiency by segmenting the users by zone in a pre-boarding line first.  Now everyone isn't standing around in one big mass, blocking the boarding line and rushing the line as soon as their block is called.  It's a hybrid of block and WILMA (which is what the Steffen method is) but not as granular as the Steffen method and adds an extra layer of organization at the gate to account for real-world considerations.


Lesson Learned on SharePoint Service Applications

Posted by Charles Chen

If you're setting out on writing your own SharePoint service applications, there is an important lesson that you should keep in mind (instead of learning it the hard way): ensure that all of your proxy, application proxy, service, service application, and service instance classes have public parameterless (default) constructors.

Otherwise, you'll have a heck of a time starting, instantiating, and uninstalling services with lots of MissingMethodExceptions and "{class} cannot be deserialized because it does not have a public default constructor" error messages.

Oddly enough, one thing I've learned from this is that the STSADM commands are often more "powerful" than the equivalent Powershell commands.  For example, Remove-SPSolution, even with the -force parameter, still failed with the aforementioned exceptions.  On the other hand, stsadm -o deletesolution {name} -override seemed to work fine.  Puzzling, for the moment, but it got the job done.  Similarly, stopping a service application that's AWOL (stuck on the processing screen) can be accomplished with stsadm -o provisionservice. Deleting it can be done using stsadm -o deleteconfigurationobject (though this one does seem to have side effects...).

Seems that Powershell is still a second class citizen when it comes to basic SharePoint command-line management.

But in any case, if you set out building your own service applications (<rant>and damn it Microsoft, can't you put some better examples out there?!  Even the few that are out there are convoluted, missing key details, hard to follow...</rant>), be sure to include public, default, parameterless constructors.

Filed under: .Net, Rants, SharePoint No Comments