Datasets without databases

How do you access non-relational data (such as a disk directory) with data-aware controls? Rely on a key element of Delphi: the TDataSet class.

By Marco Canty

A favorite subject, of late, among Delphi programmers is the merit of ADO.  During the last couple of weeks, friends have been praising the power of  ADO to access data through OLE-DB providers you can write. For example, you can access mail folders or the Windows 2000 Active Directory using the ADSI API. This struck me, not because I've decided to write an OLE-DB provider in Delphi, but because I've realized that Delphi already has a much more powerful tool than ADO offers: the TDataSet class of the VCL.

Developers use BDE or ADO as database engines to abstract access to data in different formats and from database servers. Although both approaches are used in Delphi, the real abstraction Delphi provides for data access is the TDataSet class. This class provides a high-level interface to the data used by various data-aware components. Since Delphi 3, this structure has really been opening up. Borland has begun to introduce TDataSet-derived classes (including TClientDataSet, TADODataSet, and TIBDataSet for InterBase support), as have a number of third-party vendors.

Data diversions

The custom dataset I wrote is capable of reading file data and allowing a user to display it in a data-aware component, as shown in the following figure:

You might have heard that writing a custom dataset in Delphi is a lot of work. It generally is, mainly because documentation on the topic has always been sparse. The three books I know of that address custom dataset writing are Using Delphi 3, Delphi Developer's Guide, and my Delphi Developer's Handbook. Also, The Delphi Magazine has featured two articles on the subject. Each of these references provides a complete description of the development of a custom dataset.

Writing a complete database-oriented custom dataset can be tricky. Writing a simple one to access a directory, however, is something you can easily do once you grasp the basics of dataset programming. Though the component I've written is too detailed to fully describe here, I can give you an overview of the key concepts.

A  custom dataset has to override over 20 virtual abstract methods of the base class, and it generally overrides some additional virtual methods as well. To save some extra coding in the future, I've split my work into two steps. The first is the TListDataSet component, a dataset supporting navigation among the data of a TObjectList. This is one of the new Delphi 5 container classes, which only slightly differs from TList. It uses objects instead of pointers (nothing really different, from a technical point of view) and has the ability to destroy the objects in the list when the list is destroyed or cleared.

Dataset basics

In short, the TListDataSet class contains a list of objects created in the constructor and destroyed by the destructor. The dataset works with record buffers, which store the item numbers in the Index field:

type
  PRecInfo = ^TRecInfo;
  TRecInfo = record
    Index: Integer;
    Bookmark: Longint;
    BookmarkFlag: TBookmarkFlag;
  end;

These records are created in the AllocRecordBuffer method, cleared in InternalInitRecord, and destroyed in the FreeRecordBuffer method. It is a responsibility of the dataset class to determine how many buffers to allocate and call these three methods. The bookmark is basically the same value as the Index, so I could have skipped one of the two. It is handled by the GetBookmarkData and SetBookmarkData methods, while GetBookmarkFlag and SetBookmarkFlag read and write the final field of the TRecInfo structure above. The GetRecNo and SetRecNo methods simply work on the number of the current list item, name FCur, plus one. (The one is added because the record number should start from one while the internal counter is zero-based.)

The only two methods with code are GetRecord and InternalOpen. GetRecord browses the dataset, moving back and forth while simultaneously placing data into the record buffer (in practice, only the list index plus the bookmark information). Here's the complete code for this method:

function TListDataSet.GetRecord
  (Buffer: PChar; GetMode: TGetMode;
  DoCheck: Boolean): TGetResult;
begin
  Result := grOK; // default
  case GetMode of
    gmNext: // move on
      if fCurrent < fList.Count - 1 then
        Inc (fCurrent)
      else
        Result := grEOF; // end of file
    gmPrior: // move back
      if fCurrent &mt; 0 then
        Dec (fCurrent)
      else
        Result := grBOF; // start of file
    gmCurrent: ; // nothing to do
  end;

  if Result = grOK then // read the data
    with PRecInfo(Buffer)^ do
    begin
      Index := fCurrent;
      BookmarkFlag := bfCurrent;
      Bookmark := Integer (fCurrent);
    end;
end;

The InternalOpen method performs a standard sequence of operations: initialize the field definitions, create the field objects, bind them, and initialize the internal structures. It also calls an additional virtual function I've defined, ReadListData, which stores data in the list. This operation will be performed later by the specific subclass.

procedure TListDataSet.InternalOpen;
begin
  // initialize field definitions 
  // and create fields
  InternalInitFieldDefs;
  if DefaultFields then
    CreateFields;
  BindFields (True);

  // read directory data
  ReadListData;

  // initialize
  FRecordSize := sizeof (TRecInfo);
  FCurrent := -1;
  BookmarkSize := sizeOf (Integer);
  FIsTableOpen := True;
end;

Classic derivations

Because I am dealing with a list of objects, the first thing to do in the derived class is to define those objects. In this case, I am working with file data extracted by a TSearchRec buffer by the TFileData class constructor:

TFileData = class
  public
    ShortFileName: string;
    Time: TDateTime;
    Size: Integer;
    Attr: Integer;
    constructor Create(var FileInfo: 
      TSearchRec);
  end;

constructor TFileData.Create
  (var FileInfo: TSearchRec);
begin
  ShortFileName := FileInfo.Name;
  Time := FileDateToDateTime 
    (FileInfo.Time);
  Size := FileInfo.Size;
  Attr := FileInfo.Attr;
end;

This constructor is called within the ReadListData method, mentioned earlier, to store data for the files of the current directory:

procedure TDirDataSet.ReadListData;
var
  Attr: Integer;
  FileInfo: TSearchRec;
  FileData: TFileData;
begin
  // scan all files
  Attr := faAnyFile;
  FList.Clear;
  if SysUtils.FindFirst(fDirectory, 
    Attr, FileInfo) = 0 then
  repeat
    FileData := TFileData.Create(FileInfo);
    FList.Add(FileData);
  until SysUtils.FindNext(FileInfo) <> 0;
  SysUtils.FindClose(FileInfo);
end;

At this point I need to perform two more steps. First, define the fields of the dataset, which in this case are fixed and depend on the available directory data:

procedure TDirDataset.InternalInitFieldDefs;
begin
  // TODO: set proper exception...
  if fDirectory = '' then
    raise Exception.Create('Missing directory');

  // field definitions
  FieldDefs.Clear;
  FieldDefs.Add('FileName', ftString, 40, True);
  FieldDefs.Add('TimeStamp', ftDateTime);
  FieldDefs.Add('Size', ftInteger);
  FieldDefs.Add('Attributes', ftString, 3);
  FieldDefs.Add('Folder', ftBoolean);
end;

Second, move the data from the object of the list referenced by the current record buffer (the ActiveBuffer value) to each field of the dataset, as requested by the GetFieldData method:

function TDirDataset.GetFieldData 
  (Field: TField; Buffer: Pointer): 
  Boolean;
var
  FileData: TFileData;
  Bool1: WordBool;
  strAttr: string;
  t: TDateTimeRec;
begin
  FileData := fList 
    [PRecInfo(ActiveBuffer).Index] 
    as TFileData;
  case Field.Index of
    0: // filename
      StrCopy(Buffer, 
        pchar(FileData.ShortFileName));
    1: begin // timestamp
      t := DateTimeToNative(ftdatetime, 
        FileData.Time);
      Move(t, Buffer^, sizeof(TDateTime));
    end;
    2:  // size
      Move(FileData.Size, Buffer^, 
        sizeof(Integer));
    3: begin // attributes
      strAttr := '   ';
      if (FileData.Attr and 
        SysUtils.faReadOnly) > 0 then
        strAttr[1] := 'R';
      if (FileData.Attr and 
        SysUtils.faSysFile) > 0 then
        strAttr[2] := 'S';
      if (FileData.Attr and 
        SysUtils.faHidden) > 0 then
        strAttr[1] := 'H';
      StrCopy(Buffer, pchar(strAttr));
    end;
    4: begin // folder
      Bool1 := FileData.Attr and 
        SysUtils.faDirectory > 0;
      Move(Bool1, Buffer^, sizeof(WordBool));
    end;
  end; // case
  Result := True;
end;

The tricky part in writing this code was figuring out the internal format of dates stored within date/time fields. This is not the common TDateTime format used by Delphi, but one I can convert to using the DateTimeToNative function. Moving strings and integers is simple. The attributes codes (H for hidden, R for read-only, and S for system) are extracted from the related flags, used also to determine if a file is actually a folder.

Demo delight

With this dataset available, building the demo program was simply a matter of connecting a DBGrid component to it and adding a folder selection (an old-fashioned component still available in Delphi). 

Writing the component wasn't simple, but I doubt I could have written an OLE-DB provider in less time. Writing a custom dataset, although complex, can be managed. I got faster with practice; I built my fourth one quite quickly. To make things easier, use the base class presented in this article as a framework for your own datasets based on lists. The only thing I've skipped is adding support for editing, inserting, and deleting the records, which would require considerable extra effort. I avoided this by defining the dataset as read-only (technically you have to return False from the GetCanModify method).

The complete source code for this custom dataset can be found on my Web site: feel free to use it as a starting point of your own work, and (if you are willing) share the results with me and the community.