# Thursday, April 26, 2007

Yesterday Microsoft released BizTalk Services CTP to the public. This one little CTP changes everything. Honest.

In the Beginning

It used to be very hard to build a distributed application. The pioneers in this field were Napster, Seti@Home and ICQ. To make applications like this work, you needed to have clients identify themselves and a message relay on the back end. Think of a switchboard for your applications telling user 1 how to communicate to user 2. If you wanted to build these connected systems, it would require a lot of infrastructure-a tremendous amount if you your application became popular. This has always been the barrier to entry. 

The Enterprise Service Bus

As time passed and web services came on to the scene, things got easier. A lot of the hard part of the pluming was taken care of; transport was easy via HTTP and speaking the same language was easy with SOAP. As time moved on and things got easier, people started pushing Web Services to the limit and the vendors started to really support building connected systems. After nearly a decade of XML and SOAP, most developers take it for granted. Enterprises now rely on this technology.

Enter the Enterprise Service Bus (ESB). ESB is an acronym (we always need a TLA, don’t we?) that is hot right now. The notion is that you have a set of shared services in an enterprise that act as a foundation for discovering, connecting and federating services. This is the natural evolution of the technology, as enterprises rely on the technology and themselves grow more federated, enterprises will standardize discovering, connecting and federating services.

Internet Service Bus

As Clemens argued yesterday, the release of the BizTalk Services CTP creates the first Internet Service Bus. Clemens says:

Two aspects that make the idea of a "service bus" generally very attractive are that the service bus enables identity federation and connectivity federation. This idea gets far more interesting and more broadly applicable when we remove the "Enterprise" constraint from ESB it and put "Internet" into its place, thus elevating it to an "Internet Services Bus", or ISB. If we look at the most popular Internet-dependent applications outside of the browser these days, like the many Instant Messaging apps, BitTorrent, Limewire, VoIP, Orb/Slingbox, Skype, Halo, Project Gotham Racing, and others, many of them depend on one or two key services must be provided for each of them: Identity Federation (or, in absence of that, a central identity service) and some sort of message relay in order to connect up two or more application instances that each sit behind firewalls - and at the very least some stable, shared rendezvous point or directory to seed P2P connections. The question "how does [MSN] Messenger work?" has, from an high-level architecture perspective a simple answer: The Messenger "switchboard" acts as a message relay.

Changing Business Models

In order to build distributed applications (and make money!) you have to scale to support the load your users and customers will add. This forced businesses to spend a disproportionate amount of money on hardware and too much time building the plumbing software.  Let’s take my business Corzen for example. Corzen collects specific data from the internet via spidering (like Google.) Corzen then crunches the data: we de-dupe it, aggregate it, match it with Dun & Bradstreet and US Govt data, and then apply some statistical models. We then deliver this “crunched” data to customers on a weekly basis.

The value add is the “crunching” of the data. Where do you think 75% of my technology budget goes to? You guessed it, the spidering infrastructure. We spider 24X7 and collect between 4-10 million records each week from spidering. As you can imagine our customers want more and more sites to be spidered and more frequently. We spent a year building an amazing XML based, queue and batch based, distributed application with WCF.  We can load URLs into a queue and then go to a web page where we can send that job to several spider servers via WCF. Our spider servers are low cost Virtual Private Servers on ISPs around the world running a simple spidering engine that uses reflection to dynamically compile C# code instructions and apply RegEx patterns. When Corzen was young, we manually started these engines via RDP. As we scaled, we had to build this system.

How does an Internet Service Bus change the Corzen business model and cost structure? Every few weeks we get new requests by customers to add more sites to our list of sites and we have to add spidering capacity. Since spidering is so basic, I have always wanted to have our customers spider for us (or fan out the spidering to 3rd parties for a small fee). I can then offer our customers a discount on their monthly fees based on the amount of spiders they will run for us.

This is win win win, they get more data faster, and I lower my overhead and pass that savings on to the customer. This drastically changes my focus. I spend 50% of my time managing the spidering, worrying about capacity, and expanding the spidering infrastructure. Corzen’s cost structure changes as does the relationship with our customer-we become partners in the data acquisition “plumming” and Corzen can focus on the analytics-what the customers really want us to do. In other words we get out of the raw material business (spidering) and focus on the manufacturing of the product (the analytics.) Corzen was recently acquired by a company that does spidering as well, but not the analytics. This is an exciting move since we can join forces with the spidering capacity and Corzen can focus more on the analytics.

Today, the problem with having our customers do spidering (or paying you on your idle time to do some spidering for us) is simple. With over 100 customers, you try getting our WCF application to work on all of their servers through our firewall  and their firewall. BizTalk services solves this problem. BizTalk services will provide a globally addressable name for Corzen’s service and securely expose that service to the Internet from behind a firewall or NAT as shown here.

As Dennis argues:

Use the Relay at http://connect.biztalk.net. We’ve shipped an SDK with a few samples showing you how you use the relay and identity services together. If you’re familiar with Windows Communication Foundation, you’ll find this trivial to use (by design!). Basically, your service opens at a URI on the connect.biztalk.net machines. Then a client connects to that URI and can start sending messages. We don’t want to be in the way of your app, so our relay will immediately try to establish a direct connection between clients. More details on this how this all works in a later post. Here’s a quick diagram that describes it at a high level.


Microsoft vs. Google

Everyone always likes to compare the “war” between Microsoft and Google. Maybe it is Google’s stock price or the popularity of its search engine or Gmail system. I look at the companies as light years apart. At its core, Google is an advertising company (highlighted by its recent acquisition of DoubleClick.) Sure Google will expose some APIs for developers, however, everything it does from Gmail to search is to gain more eyeballs for its ads.

Microsoft is a company about selling Windows. What has made Windows so popular is that Microsoft gives developers amazing tools to build applications around Windows, Office and the Internet. The strength of Microsoft is the developer community surrounding its products. You always hear about the next great thing that is going to “take Microsoft down.” The only thing that will take Microsoft down is a company with a compelling platform that also provides tools for developers to create applications on that platform.

The latest thing to come to take Microsoft down is Software as a Service. Think Google Spreadsheet. Businesses will all use the Spreadsheet in the sky and store its data on Google’s servers. Ditto with Gmail, why bother with Exchange?

Microsoft does have a cool differentiating factor: its hybrid approach. Microsoft is offering Software + Services.  As I said in eWeek, with such a huge commitment to the OS and other installed software already, Microsoft is actually in a position to deliver software and a service on top of it.

The marketplace wants a hybrid solution and Microsoft is the only one who can deliver it in the short to medium term, giving Microsoft a competitive advantage. Everyone thought Google Docs would kill Office but in reality, Google docs are cool but Enterprises have issues today with using it offline and inside of a browser (copy and paste is strange, so is right clicking in a cell). Personally I use Google Spreadsheet to keep track of simple things but Microsoft Excel to do the more processor intensive operations.  In addition, I work offline a tremendous amount as well don’t trust Larry and Sergey to store my very sensitive documents.

Think of Excel as software + services. Excel can be sold and run on your computer. You can store your documents locally or up in the cloud. Processor intensive operations can utilize your local super-fast Pentium 1million Processor and your 2 GB of RAM. Collaborative efforts can be handled by the cloud.  Additional services like statistical number crunching or anything that needs to be distributed can be handled by the cloud. Multiple editors and viewers, in the cloud. You get the point.

This is where BizTalk Services come in. It is an early way for developers to deliver Software plus Services.

The Future of Business

As I said, this changes everything. We can all agree that distributed applications are the future. In order to make money you have to scale to support the load your users and customers will add. This forced businesses to spend a disproportionate amount of money and focus on technology (and pay CTOs way too much!)

We are so focused on technology that CEOs and Venture Capitalists are desperately trying to learn about the basics of technology-time they should be spending working through business models and looking for competitive advantages.

BizTalk services and ultimately all of the Software plus Services (including other vendors, not just Microsoft) will change the way we do business in 5-10 years. Imagine if we had to run a switchboard to run our phones in the office? So an oil company or a bank would have to develop the technical expertise to run the phones. This infrastructure is solved by the phone company (and now VoIP!) In the future, businesses will only have to focus on their core businesses and most software will run locally with services up in the cloud, drastically reducing the investment in core IT infrastructure internally. It’s a brave new world out there.

posted on Thursday, April 26, 2007 3:33:17 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0] Trackback
# Thursday, April 12, 2007

Thursday, April 19, 2007
Interfacing External Hardware Using Managed Code and Microsoft Robotics Studio

Subject:  While developers write code to build software every day, not often are they exposed to code that drives and interfaces hardware. This session will attempt to bridge that gap and show how .NET can be used to effectively interface several hardware devices, including an RFID reader and tags, Phidget control boards with a variety of sensors, and a servo controller. Additionally, Microsoft Robotics Studio will be introduced along with a demonstration showing how one can control some of the hardware previously listed in an environment where concurrency and performance are key.
Speaker:  Brian Peek , ASPSoft

Brian Peek is a recognized .NET expert with over 6 years experience developing .NET solutions, and over 9 years of professional experience architecting and developing solutions using Microsoft technologies and platforms. Along with .NET and its associated languages, Brian is particularly skilled in the languages of C, C++ and assembly language for a variety of CPUs. He is also an expert in a variety of technologies including web development (ASP.NET, ASP, Javascript, HTML, XML, etc.), document imaging, GIS, graphics, game development, and hardware interfacing. Brian has a strong background in developing applications for the health-care industry, as well as developing solutions for portable devices, such as tablet PCs and PDAs. Along with Jonathan Goodyear, he co-authored the book "Debugging ASP.NET" published by New Riders. He is also a member of MSDN's Coding4Fun writing team, contributing articles on a monthly basis.
Date:  Thursday, April 19, 2007
Time:  Reception 6:00 PM , Program 6:15 PM
Location:   Microsoft , 1290 Avenue of the Americas (the AXA building - bet. 51st/52nd Sts.) , 6th floor
Directions: B/D/F/V to 47th-50th Sts./Rockefeller Ctr
1 to 50th St./Bway
N/R/W to 49th St./7th Ave.


posted on Thursday, April 12, 2007 3:05:42 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0] Trackback
# Wednesday, April 11, 2007

Bill Zack will be transitioning to a new role at Microsoft.  He has accepted a position as Architect Evangelist in the Developer and Platform Evangelism (DPE) group at Microsoft.  In his new role he will be doing a series of day-time Architecture briefings on enterprise topics in New York at Microsoft. On April 12th Bill will be doing one on Microsoft and SAP Interoperability as part of Microsoft's arcStream program. This is a series of live presentations combined with access to on-line experts from Microsoft. 

If you are currently using SAP and Microsoft technology in your company but have not integrated them you should find attending this meeting very informative.  If you use SAP where you work or are just interested in finding out about this subject feel free to attend. The presentation will be held at Microsoft’s New York City offices. You are invited to join.

arcStream provides a constant flow of technical information as well as networking opportunities for enterprise, application, systems and aspiring architects living and working on the east coast.

For more information see http://www.arcstreameast.net/arcStreameast.

In addition Goksin Bakir will be starting as a DE in the Middle East and Africa region. While I hate Goksin for leaving me as the last of the club to be independent from Microsoft (first Malek, then Clemens!), it is a GREAT move for him and even better for the region. I can't wait to speak at even more Middle East and Africa events!

posted on Wednesday, April 11, 2007 11:13:17 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [0] Trackback
# Tuesday, April 3, 2007

The joke goes that whenever Microsoft says to you “Thanks for your feedback” you can usually translate that to “Go Fuck Yourself.” Not this time. We gave our feedback and Microsoft listened. Today they announced that they will be making Expression Web available starting today to all MSDN Premium subscribers. This is great for developers! (and great for Microsoft.)

Read about it here.


posted on Tuesday, April 3, 2007 11:54:42 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [0] Trackback