Rants Tagged with “Oslo”
1 2 > >> (Total Pages: 2/Total Results: 13)
Chris Sells and Craig Andera are trading blogs about what is and isn't a DSL. Craig has issue with the following from Chris' blog:
Here’s what he says, abridged:
(when (and (< time 20:00)
(timing-is-right))
(trade (make-shares 100 x)))
IMO, that's not a DSL -- that's just a set of function calls in an existing language.
Craig answered back (a section, possibly completely taken out of context):
A set of function calls in an existing language is generally preferable to a completely separate language. Particularly if it’s the language that you’re using to write the rest of your system.
While the concept of domain specific languages is different things to different people, I fundamentally think that the "Little Languages" (Jon Bentley, Little languages, Communications of the ACM, 29(8):711--21, August 1986.) philsophy applies. As far as I am concerned that means the languages should be simple enough to write by hand and clear enough that non-technical stakeholders can understand them. Why? Because for my money, a DSL is useful when it can help you ensure that the intent of a system is the same as the code that runs a system. And you cannot be sure of intent unless someone can assert the rules are correct.
I am sure I am missing some of the point of DSLs and feel free to educate/flame me, but for me the gain of building a DSL (even a very simple one) is that it can be validated against requirements much easier than typical code. In fact, it should be editable by stakeholders as well (perhaps not all stakeholders, but some).
If your DSLs are all based on language semantics (e.g. internal versus external DSLs I think its referred to by Fowler) then it makes them only readable to developers. Sounds like the use-case was for a rules-engine not a DSL.
In Part 1 of this series I showed you how to create database schema with MSchema. Then in Part 2 I showed you how to use MGraph to create data to store the in the database along side database schema. In this last part, I will show you how to use the M tools to put the schema and data in the database. I use these tools to build my database during a build script.
The first thing we need to do is to compile the schema. You can do this with the m.exe tool that ships with the Oslo SDK. While many of the demo's I've seen have pushed the data into the Repository, I want to push it into a generalized database. (My site isn't ready to be build on top of Oslo since its going to be released quite soon so I am just using the MSchema stack to help me build the database). When we compile the schema we can tell the tool type of target we're going to aim for. In our case we will target TSQL instead of the repository. So the first call in my database build script is to build the schema like so:
m.exe students.m /p:image -t:TSQL10
This tells the m.exe compiler to build an image (in this case an .mx file) based on our schema (the students.m file). This stop only builds the database, we now want to push it into the database. We do this with the mx.exe tool:
mx.exe /i:students.mx /db:Training /ig /f /verbose
This part of the script uses the compiled version of the schema (the student.mx file created by our last step and insert it into a database called Training. We could specify a server and security credentials but since I am building it with the local database and integrated security, we didn't need to specify these. The /ig tells the database to ignore missing dependencies and try to build the database from the schema from scratch. The /f flag says force the schema (including deleting existing elements with the same name). Now we have the schema in the database. But what about the data we created?
To insert the data, our studentsdata.m file doesn't know about the schema so we need to hint it to the compiler:
m.exe studentsdata.m /p:image -r:students.mx -nologo -t:TSQL10
This call looks the same as the first except we reference our compiled schema file with the -r flag. Like before this creates a compiled version of the data we're putting in the database. So you probably have already guessed that we use the mx.exe tool to insert the data into the database:
mx /i:studentsdata.mx /db:Training /ig /f /verbose
This looks just like our schema database insertion, but we're going to use the compiled data file instead. Because we compiled it with the reference to the compiled schema file, the new .mx file has everything it needs to insert itself into the database as rows in the database.
Hopefully this series has helped you understand how the M languages and the tools can work together to manage schema creation for projects. There are a lot of other features that we haven't discussed (like versioning) that are supported but i'll leave that for another day.
Do you think you'll use MSchema on any upcoming project?
If you have read Part 1 of this series, you have seen how to create types, collections and constraints using MSchema. Along with most database schemas, I find the need to create some amount of data to go along with those schemas. For my current project (our new web site), I wanted to have some small set of data that included some basic workshops and locations.
To create the data we can use the MGraph language. MGraph is related to MSchema but is a way of describing concrete instances of data. To create an instance:
module Training
{
Workshop
{
{
Name = "Silverlight Workshop",
Description = "A three day workshop to teach Silverlight 2.",
Length = 3,
MaxAttendees = 16,
DefaultPrice = 1800.00
},
{
Name = "Advanced Silverlight",
Description = "Two-day course teaching advanced Silverlight topics.",
Length = 2,
MaxAttendees = 16,
DefaultPrice = 1200.00
}
}
}
The syntax looks similar to a mix between the C# object initialization syntax and JSON. The module matches the same module that the schema was defined in. Then it is the extent name (Workshop in this case) then an array of objects (notice the types defined inside curly braces). This can be used to generate insert statements. That's easy to do with actual insert statements, so why learn this format at all?
Foreign keys are what make this fun. If we wanted to add an event (who has a reference to a Workshop) we would need to be able to tell the MGraph language how to make the reference. I innocently thought of this approach:
Event
{
{
WorkshopId = 1,
EventDate = 2009-02-04
}
}
I assumed that I could set the id's manually then use them...to my surprise MGraph handles that better that I guessed. how so? The creation of the instances can be named. So if we change the workshop creation to use names like so:
Workshop
{
SilverlightWorkshop {
Name = "Silverlight Workshop",
Description = "A three day workshop to teach Silverlight 3.",
Length = 3,
MaxAttendees = 16,
DefaultPrice = 1800.00
},
AdvancedSilverlight {
Name = "Advanced Silverlight",
Description = "Two-day course teaching advanced Silverlight topics.",
Length = 2,
MaxAttendees = 16,
DefaultPrice = 1200.00
}
}
Now that the workshops are named, we can define our events with the names:
Event
{
{
WorkshopId = Workshop.SilverlightWorkshop,
EventDate = 2009-02-04
}
}
Now when we create the event, we can use the workshop (notice the Workshop is prefixed) to say that the workshop for this event is the one we created above. M takes care of the rest.
In the next part, we will show the command-line tools you can use to store the schemas and data in the database.
As most of you know, I run a small training company. We are in the midst of a re-write of our main website to allow for more cohesive registration and information about our classes. For this project, I am completely re-creating my database schema (as the old one was a bit 'off the cuff'). What a great opportunity to try out MSchema to build the new schema.
I am going to show you some features in three parts. In this first part I will show you how to define some interesting data types in MSchema. In the second part I will show you how to define your static or test data in MGraph that can be inserted into the database along with your schema. Then finally in the third part I will show you how to use the command-line tools to build your schema from the M files. Onto the first part!
I'll admit, I hate visual editors. In fact, in his PDC talk, it was as if Doug Purdy was talking to me when he said we have an emotional connection to our text editors. I have spent some time with emacs.NET IntelliPad and so far I am impressed. It is taking some getting used to but the extensibility is pretty impressive.
I like the way that MSchema forces me to create a type then push it into an extent (e.g. to make it a table). For example, I started with a Workshop type that defined a training topic:
module Training
{
type WorkshopDef
{
WorkshopId : Integer32 = AutoNumber();
Name : Text#100;
Description : Text?;
Markup : Text?;
Length : Integer32;
MaxAttendees : Integer8;
DefaultPrice : Decimal19;
} where identity WorkshopId;
}
I start with a module which just creates something like a namespace for our types and extents. This module will become the Schema in SQL Server for our tables and views. Next, I created a type with the fields I wanted. You notice that I post-pended "Def" on the type. This is mostly just a convention I made up so that I could name the collections singular (we'll see more about that later when we see the generated schema). You can see the fields for my type and the constraints I am specifying (e.g. Text#100 is the same as saying Text where value.length <= 100). Like in C#, the question mark says that the field is nullable. Finally, I specify that the type has an identity using the field(s) that make up my primary key.
At this point all I have is a type, no tables or view or anything really. I need to extend this into a collection:
Workshop : WorkshopDef*;
This extent simply says create some place to hold things that look like WorkshopDef and call it Workshop. The asterisk at the end of WorkshopDef implies zero or more (like in regular expressions). This set of schema would yield a small amount of SQL:
create schema [Training];
go
create table [Training].[Workshop]
(
[WorkshopId] int not null identity,
[DefaultPrice] decimal(19,6) not null,
[Description] nvarchar(max) null,
[Length] int not null,
[Markup] nvarchar(max) null,
[MaxAttendees] int not null,
[Name] nvarchar(100) not null,
constraint [PK_Workshop] primary key clustered ([WorkshopId])
);
go
You can see the schema created closely matches our type definitions. That's the cool factor for me. Let's do something more interesting. For each Workshop I have a list of Topics (and eventually sub-topics). These are bullet point items that show what is covered in a workshop. So I can add a new type like so:
type TopicDef
{
TopicId : Integer32 = AutoNumber();
WorkshopId : WorkshopDef;
Name : Text#100;
} where identity TopicId;
Note that when I define that I want a WorkshopId to be a reference to another type, I specify it as the type I am referring to. This means in our extent for Topics we can create the foreign key like so:
Topic : TopicDef*
where item.WorkshopId in Workshop;
This essentially says that we want topics, but be sure that WorkshopId are in the Workshop table (the foreign key). The next thing to show is the real power of constraints. If you saw the Doug Purdy talk you probably saw that you could create a constraint for simple data like so:
type AttendeeDef
{
AttendeeId : Integer32 = AutoNumber();
FirstName : Text#50;
LastName : Text#50;
MiddleName : (Text#50)?;
DateRegistered : Date;
HasCancelled : Logical = false;
Phone : (Text#50)?;
Email : (Text#100)?;
TShirtSize : Integer32 where value <= 5;
} where identity AttendeeId;
In the TShirtSize, notice how I am constraining the value to a range of numbers from zero to five. This may be a magic way of translating from small to 3x. The SQL actually works as it creates a check constraint:
create table [Training].[Attendee]
(
[AttendeeId] int not null identity,
[DateRegistered] date not null,
[Email] nvarchar(100) null,
[FirstName] nvarchar(50) not null,
[HasCancelled] bit not null default 0,
[LastName] nvarchar(50) not null,
[MiddleName] nvarchar(50) null,
[Phone] nvarchar(50) null,
[TShirtSize] int not null,
constraint [PK_Attendee] primary key clustered ([AttendeeId]),
constraint [Check_Attendee] check ([Training].[Check_Attendee_Func](
[AttendeeId],
[DateRegistered],
[Email],
[FirstName],
[HasCancelled],
[LastName],
[MiddleName],
[Phone],
[TShirtSize]) = 1));
go
The Check_Attendee_Func is a generated function to test for constraints (including the size of TShirtSize. But that is making TShirtSize a magic number. I don't like magic numbers. Let's create something like an enumeration instead:
type TShirtSizeEnum
{
"Small",
"Medium",
"Large",
"X-Large",
"2X",
"3X"
};
type AttendeeDef
{
AttendeeId : Integer32 = AutoNumber();
FirstName : Text#50;
LastName : Text#50;
MiddleName : (Text#50)?;
DateRegistered : Date;
HasCancelled : Logical = false;
Phone : (Text#50)?;
Email : (Text#100)?;
TShirtSize : TShirtSizeEnum;
} where identity AttendeeId;
The new TShirtSizeEnum type can be used to constrain the value of the TShirtSize field. How do we do that? We simply tell MSchema that we are going to store a TShirtSizeEnum in that field and it creates it in the generated SQL.
Just a couple of days ago I did the Atlanta stop of the MSDN Developer Conference where David Scruggs and I did a "Lap Around Oslo" talk that seemed to go very well. If you are lucky to have this conference coming to your town, its well worth the $99 investment. Great topics and great speakers (especially if you missed the content at the PDC).
In preparation for the talk, I spent a lot of time looking at the demo script that we were supplied. It was well thought-out and tried to touch on many aspects of the Oslo platform. After the talk we got a number of good questions a about Oslo, Model Driven Development and Domain Specific Languages. That was great. The only frustration is that some of the students got stuck on the concepts in MSchema. They seemed to be more interested (or more importantly could see the value) in using MSchema to define their schemas and data. My fear here is that there were getting caught up in the details instead of seeing the big picture (which to me is Models and DSLs).
This isn't the first technology talk that to confused and obfuscated. LINQ is my favorite example. In most of the early LINQ demos I saw, LINQ to SQL was used as an example of the power of language integration of a query language. The problem is that people left the talk thinking (and many still think) that LINQ is an ORM over SQL Server. This causes two problems, either attendees that miss the message leave thinking that the technology doesn't apply to them (i.e. I already have an ORM) or that they focus on the part of the technology that they understood well (LINQ to SQL has a big following even if Lambda's and LINQ in non-db hasn't gotten as much traction as it might).
Applying this learned lesson to Oslo, I sincerely hope that attendees heard the message of model driven systems and domain specific languages. These two technologies are not a new invention (after all, "Little Languages" was coined in 1994), but Oslo does make it very approachable. Writing a DSL grammar with MGrammar is simpler than my earlier attempts with other technologies. These represent an opportunity to shift the way we develop software. I hope that message is being heard loud and clear.
What's your take?
UPDATE: Added a couple of screenshots to clarify what the coloring does.
I am spending quite a bit of time this week writing my own MGrammar using the Oslo SDK. When writing a MGrammar, you can use something like attributes to help Intellipad know how to color your text to help make it more readable. For example,
@{CaseSensitive[false]}
language PricingLanguage
These attributes are used by different parts of the Oslo stack to hint the system with additional information (like Attritubes in .NET). The one that I was particularly interested in was the Classification attribute. This attribute allows you to tell IntelliPad how to format the part of the language. For example:
@{Classification["Comment"]}
token Comment = "//" ^("\r" | "\n")+;
The only two types of classifications that I could find or guess were seen in the PDC videos: "Comment" and "Keyword". The MGrammar and IntelliPad docs didn't mention anything about classifications so I asked up on the Oslo Forums. While it isn't documented anywhere, I was told about a configuration file call ClassificationFormats.xcml that resides in the Settings folder of Intellipad (usually c:\Program Files\Microsoft Oslo SDK 1.0\Bin\IntelliPad\Settings). This lead me to two important pieces of information: a fuller list of classification types (i.e. Strings, Delimiter, Type, etc.) but also showed me how to customize the coloring as the default Intellipad coloring is starkly monochromatic. This file is a simple XAML file that contains individual classifications and how they are formatted:
<ls:ClassificationFormat Name='Keyword'
FontSize='13'
FontFamily='Consolas'
Foreground='#FF333333' />
Each of these can be changed to specify the color (Foreground) and even the font and size (if you want unreadable code). You can edit this file and just use the Shift-Alt-F5 hotkey in Intellipad to reload the settings to tweak the colors as you like.
For example, I've edited the Keyword and Operator colors to show blue and red respectively. So that in my MGrammar if I use the Classification attribute to specify parts of my DSL are Keyword tokens or Operator tokens as shown here:
Then my language is colored based on my ClassificationFormats.xcml file:
Nice...
Coming December 16, 2008, i'll be at the Atlanta stop of the MSDN Developer Conference to see the PDC content. These events will give you an opportunity to see the Azure platform, Windows 7, Silverlight, F# and even Oslo.
I will be presenting the "Lap Around Oslo" with David Struggs. If you want to learn about the newest stuff coming out of Redmond or just want to show up and argue with me about Domain Specific Languages, show up at the event.
For the nominal $99 fee you get admission, food and the chance to win some fun prizes.
See you there!
Coming December 16, 2008, i'll be at the Atlanta stop of the MSDN Developer Conference to see the PDC content. These events will give you an opportunity to see the Azure platform, Windows 7, Silverlight, F# and even Oslo.
I will be presenting the "Lap Around Oslo" with David Struggs. If you want to learn about the newest stuff coming out of Redmond or just want to show up and argue with me about Domain Specific Languages, show up at the event.
For the nominal $99 fee you get admission, food and the chance to win some fun prizes.
See you there!
Now that Oslo is in a public form, I've taken time (as you probably noticed in earlier blog posts) to look at Oslo. While the Model Driven Development part of the stack is important and potentially game changing, I wanted to stop and look at the Domain Specific Language part of the Oslo stack.
I am not the only one though. The potential for building Domain Specific Languages has caused jeers and leers from different parts of the web including Martin Fowler, Frans Bouma, and Roger Asling. Aside from the whether "M" becomes a valid way to build Domain Specific Language, I am more interested in the idea behind Domain Specific Languages themselves.
Honestly I am not sure whether I am early or late to the party. My interest was peaked at a party where I was a fly on the wall with the likes of Don Box, Chris Sells, Ted Neward, Neil Ford (et al.) all discussing what it meant to be a DSL and why it was important. I've be stewing and thinking about it quietly since then to try and understand really what is means to me and the problem spaces I care about. This is what I came up with.
Business software is written to solve business problems. The people feeling the pain of the business problem are often the people running the business. The people that run a business are rarely the technical folks. Before software is written we need to understand the problem so that we can figure out how to solve the problem. That means that technical folks have to talk to the non-technical folks to understand the problem. Traditionally that is how requirements are gathered. So what happens next? Typically the requirements are written down and communicated back to the non-technical folks to see if the requirements are still correct as simply transcribing them can introduce errors in the requirements. Finally the requirements are shared with developers to implement the requirements.
This doesn't work well as well as it should.
This problem is that the two groups involved often do not speak the same language. Language is a big deal here. Every business has its own vocabulary and unless the technical people have been in the business for a long time, they don't understand that vocabulary. Even developers have their own vocabulary and we know that most people don't understand what we are saying (especially our spouses), so why should it surprise us that we often do not understand the vocabulary of the business we are working with/for?
This is exacerbated by the problem that contractors are being used to a great extent these days. It used to be that developers in an enterprise would learn their domain. Part of the process of working in a company would be learning the language of the company and how they did business. To my surprise many companies are investing in contractors to teach them their domain and then cutting them loose. This means we need to have a way get domain knowledge "out of the head" of the stakeholders and into a form that can live for longer than one product cycle.
In the recent Oslo .NET Rocks Podcast, Don Box and Doug Purdy talked about the problem of coders and encoding. If you haven't listened to the podcast yet, i'll paraphrase. We learn from the non-technical folks how to run their business, then we write it down. We take the written down version and ask a 'coder' to encode the rules in code. The essential problem here is that we can't have the non-technical folks read the 'encoded' version to figure out if it matches their intent. It also assumes that the developer understands the language of the business well enough to make sense of the written down version.
The other problem is that as the business changes, the whole cycle has to repeat. Once the documented business process is encoded, changing it requires re-encoding of the change. This is problematic.
How do Domain Specific Languages fit into problem space? I think that if we can talk with the non-technical folks and create a textual DSL (or perhaps a visual DSL) that is clear enough for the non-technical folks to read it (not necessarily write it), then we can use that to document the business problems. That would allow for the developer to use that data at runtime instead of encoding it. That means that the non-technical folks can *validate* the actual business rules (not the documentation that is being encoded) so that our systems are less prone to communication confusion. In addition, this means that as the business changes, the changes to the business rules are interpreted by the system instead of being re-encoded.
This doesn't take the developer completely out of the picture as developers still need to use the runtime information to make the system work as well as develop and maintain the DSL. Rarely will a DSL be complete. As business requirements change, the DSL will often have to change to accommodate new business rules. But that leaves the developer from having to have intimate knowledge of the business (though tertiary knowledge is still required).
At the end of the day this allows fewer points of failure between the business intent and the runtime environment. At least that is my hope.
Of course, this is not new. We build and use Domain Specific Languages all the time on projects. Every time we build a new configuration file section, XML Schema, or use one of the plethora of common DSLs (e.g. XSLT, XAML, CSV) we are using DSLs. The difference that I hope this brings in is the ability to have DSLs that are consumable by non-technical stakeholders in the business.
What do you think?
When I first grabbed the Oslo SDK, I wanted to first dive into MSchema. MSchema is a language for defining your data store and relationships between data that Oslo uses to define how to handle storage. My first attempt was to try and replicate the store model of my VideoGameStore data that i've been using to show off Silverlight and ADO.NET Data Services. My original attempt was:
module VideoGameStore
{
type Product
{
ProductId : Integer32 = AutoNumber();
ProductName : Text where value.Count() < 250;
Description : Text#250;
ReleaseDate : Date?;
} where identity ProductId, unique ProductName;
Products : Product*;
type Game : Product
{
Genre : Text#100;
Developer : Text#150;
Publisher : Text#150;
};
Games : Game*;
}
This resulted in the non-decorator table solution of:
create table [VideoGameStore].[Products]
(
[ProductId] int not null identity,
[Description] nvarchar(250) not null,
[ProductName] nvarchar(249) not null,
[ReleaseDate] date null,
constraint [PK_Products] primary key clustered ([ProductId]),
constraint [Unique_Products_ProductName] unique ([ProductName])
);
go
create table [VideoGameStore].[Games]
(
[ProductId] int not null identity,
[Description] nvarchar(250) not null,
[Developer] nvarchar(150) not null,
[Genre] nvarchar(100) not null,
[ProductName] nvarchar(249) not null,
[Publisher] nvarchar(150) not null,
[ReleaseDate] date null,
constraint [PK_Games] primary key clustered ([ProductId]),
constraint [Unique_Games_ProductName] unique ([ProductName])
);
go
The reason this didn't work is that (curly braces aside), MSchema isn't talking inheritance (as I saw it). In my model I specify that the type "Game" is 'derived' from "Product" which feels like inheritance but is actually extended the type, which is why the resulting SQL results in two tables with overlapping data.
What I should have done was:
module VideoGameStore
{
type Product
{
ProductId : Integer32 = AutoNumber();
ProductName : Text where value.Count() < 250;
Description : Text#250;
ReleaseDate : Date?;
} where identity ProductId, unique ProductName;
Products : Product*;
type Game
{
Product : Product;
Genre : Text#100;
Developer : Text#150;
Publisher : Text#150;
} where identity Product, Product in Products;
Games : Game*;
}
In this case I am creating a type that encompasses the Product type so in this case by including the Product in the type, I am creating that extension that wanted. By making this change I get a different database schema generated:
create table [VideoGameStore].[Products]
(
[ProductId] int not null identity,
[Description] nvarchar(250) not null,
[ProductName] nvarchar(249) not null,
[ReleaseDate] date null,
constraint [PK_Products] primary key clustered ([ProductId]),
constraint [Unique_Products_ProductName] unique ([ProductName])
);
go
create table [VideoGameStore].[Games]
(
[Product] int not null,
[Developer] nvarchar(150) not null,
[Genre] nvarchar(100) not null,
[Publisher] nvarchar(150) not null,
constraint [PK_Games] primary key clustered ([Product]),
constraint [FK_Games_Product_VideoGameStore_Products]
foreign key ([Product])
references [VideoGameStore].[Products] ([ProductId])
);
go
So my new Games table truly extends the Product table (as a decorator table). I have a reference to the Product in the new table (represented by the Product value which is a foreign key to the Product table).
My problem is that MSchema (and likely with the rest of the M family of languages) is that they feel like curly-braced languages (e.g. C#, C++, C, Java) so that when I am writing M I get caught up in trying to apply concepts from that world to this very different world.
At least the guys on the Oslo forum helped sort me out...
