Paul Stovell says…

Recently I have become a huge fan of LINQ, and have been doing a little work to extend LINQ which I hope to announce soon. It’s not the SQL server integration of LINQ to SQL that interests me, but the elegance of it: developers simply express a need for information, and the system can decide how best to fulfil it.

The Web Service Divide

One thing that I look forward to seeing is how LINQ will fit into typical multi-tier architectures:

 

In a scenario like the one above, we might use LINQ behind the Web Service to deal with our objects and databases. But can it be used from the Windows App? I’m not so sure. As you’re probably aware, the contracts exposed by web services are generally very “constrained”; for example:

Guid[] GetCustomerIDsInState(string state);
BankAccount[] GetBankAccounts(Guid customerId);

Now, let’s say the Windows Application wanted to list all overdrawn bank accounts. On the server, the query could be expressed like this:

BankAccount[] overdrawnAccounts =
    from account in AllBankAccounts
    where account.Balance < 0
    select account;

However, since they are using web services, they would be forced to write code that involves a number of service calls as well as client side processing:

foreach (Guid id in bankProxy.GetCustomersInState(”NSW”))
{
    foreach (BankAccount account in
        bankProxy.GetBankAccounts(customerId))
    {
        if (account.Balance < 0)
        {
            // Show the account
        }
    }
}

Or, alternatively:

BankAccount[] overdrawnAccounts =
    from customerID in bankProxy.GetCustomerIDsInState(”NSW”)
    from account in bankProxy.GetAccounts(customerID)
    where account.Balance < 0
    select account;

Which is still going to involve many service calls. So, for efficiency, we’d probably have to push that query logic down behind the web services:

Guid[] GetCustomerIDsInState(string state);
BankAccount[] GetBankAccounts(Guid customerId);
BankAccount[] GetOverdrawnAccountsInState(string state);

This begs the question: in a multi-tier architecture, can LINQ be used on the client side?

TFS versus CRM

Web Services (including WCF, and even REST services) are usually very constrained in their operations. While this gives extra control to the people defining the services, if the services provided aren’t very well-thought-out, it can leave the client performing some very inefficient actions. The Team Foundation Server services are a good example of this.

Most TFS information is available just from one service call, but if you wanted to, say, get a list of all running builds, in all build types, in all Team Projects, you’d be in for a very chatty experience.

Get list of team projects (service call, perhaps 10 results)
  Get list of builds types (service call, perhaps 3 results per project)
    Get build detail list (service call, perhaps 30 builds per build type)
      if build.State == "Running"
         display

Given that there is only likely to be, at most, a handful of builds running at once, we must ask why it takes so many web service calls to find them.

Microsoft CRM, on the other hand, provides a very different experience. CRM offers regular web service calls, but it also provides an ability to write very powerful queries, known as FetchXML. For example, you could make a web service call to CRM passing the following XML:

<fetch mapping='logical'>
    <entity name='account'>
        <attribute name="accountid" />
        <filter type='and'>
            <condition
                attribute='createdon'
                operator='on-or-before'
                value='2007-10-17'
                />
        </filter>
    </entity>
</fetch>

This query would fetch the ID’s for any Accounts which were created on or before the 17th of October. Behind the web services, CRM transforms this into an SQL query and sends the results back as XML which you can then parse into objects. This is much more efficient, but as you can imagine, it would have taken a lot of work to implement. Although CRM’s FetchXML works over Web Services, it is very much a custom format.

(It strikes me that a LINQ to FetchXML adapter would be a fun project)

Web Service Queries

If we want to make efficient use of LINQ on all sides of the web service divide, it’s important to allow the client to write queries against the server, much like FetchXML allows us to do. But how can we do this in a way where we don’t have to come up with our own querying language?

I think the answer involves changing the way we think about web services.

The Team Foundation Server service layer says:

“Here is a list of operations you may perform.”

Whilst the CRM service layer says:

“Here is a list of abstract entities, and the operations that can be performed against them”

Note that this doesn’t make the web service layer any less of an “insulator” than in the first example. To illustrate, if you look at the CRM database schema, you’ll find that it looks quite different to the abstract “entities” that CRM exposes for querying. A translation layer sits in between to provide the abstraction necessary to protect the web service implementation from becoming coupled with the client.

Once our web services define entities and the operations that are valid upon them, it would be fairly easy to build a LINQ query provider to turn object queries into a SOAP web service call, and on the other end, to turn them back into object expressions which can then be executed on the server.

For example, the developer on the client side might write:

BankAccount[] overdrawnAccounts =
    from account in bankProxy.Accounts
    where account.Balance < 0
    select account;

Which the LINQ query provider translates into:

<Query>
  <MethodCallExpression Method="Select“>
    <MethodCallExpression.Operators>
      <ConstantExpression Name=”account” />
      <MethodCallExpression Method=”Where“>
        <MethodCallExpression.Operators>
          <ConstantExpression Name=”account” />
          <LambdaExpression>
            <LambdaExpression.Operator>
              <MemberExpression Name=”Balance” />
            </LambdaExpression.Operator>
            <LambdaExpression.Operand>
              LessThan
            </LambdaExpression.Operand>
            <LambdaExpression.Value>
              0
            </LambdaExpression.Value>
          </LambdaExpression>
        </MethodCallExpression.Operators>
      </MethodCallExpression>
    </MethodCallExpression.Operators>
  </MethodCallExpression>
</Query>

And on the web service side, the technology that provides the abstraction over turning objects to and from XML (ASP.NET or WCF) could turn the query back into an object expression to be executed.

(Of course, the serialized query would probably look quite different to the above - in fact, what I wrote above probably made no sense, but you get the idea)

Security

Executing arbitrary code from a client on the server would never be a smart thing to do, but that’s not quite what I’m proposing here.

In LINQ to SQL, for example, some expressions such as customerName.StartsWith("P"), can be translated into SQL (where CustomerName LIKE 'P%'), and the LINQ to SQL query provider is smart enough to figure this out. However, other expressions, such as File.Delete("blah.txt"), which could legally be placed within a call to a LINQ query, can’t be translated into SQL. LINQ to SQL executes as much of the query as it can on the SQL server, and is smart enough to recognize the things it can’t translate, and executes them using .NET code back on the client.

The same thing would happen with queryable web services. The WSDL provided by the web service would define entities and any operations that are valid to be executed on the server, and any part of the query that isn’t allowed to be called on the server could be executed on the client.

Summary

I am really just throwing this idea out there to see if there is interest. To summarise, what I’m proposing is:

• LINQ, and querying in general, should be usable from both the client side and the server side of a system
• We come up with a standard XML syntax around defining entities and expressing method calls upon them. Essentially, a serialized LINQ Expression Tree.
• We make use of LINQ query providers to convert to and from the XML transport mechanism.

The benefits:

• Implementing services is easier - we don’t have to imagine all the common operations that people might perform
• Consuming services is easier, and leads to a much less “chatty” use of the services

What do you think? If such a system were built into the .NET framework, do you think you would design your Web Service endpoints around it?

Filed under: Web Services, WCF