Saturday, November 20, 2004

Interop Wiki

Just stumbled upon this interop reference web site.
It's a wiki containing:
  • API signatures and type definitions
  • gotchas, tips and samples
  • alternative managed APIs
  • other community resources
It only works in Internet Explorer at the moment.

Thursday, October 14, 2004

madCodeHook

I'm using madCodeHook from madCollection to fix an ugly problem we (at the company I work for) have with some of the earlier versions of our products. We've been using JCL (the JEDI Code Library, an otherwise extremely useful and very well-written library), which used to have a bug in its CPU speed measuring code: it might cause a division by zero on faster computers. What makes things really bad is that this code is executed during initialization - which, if the division by zero occurs, prevents the applications from running.

The applications are running from CDs (and require the CDs to run). They have been working fine for years, because the bug does not reveal itself on slower CPUs, but now, as users start upgrading or buying new, faster computers, they find out that the applications won't run anymore (they won't even start). :-(

The fix I'm working on injects a DLL into every newly started process as soon as it's loaded (before initialization code is executed). The DLL inspects the process it has been injected into, and if (and only if) it positively identifies it as our application, it hooks the QueryPerformanceFrequency API call to return False, which solves the problem - the offending JCL routine will not attempt to measure the CPU speed in such case. The DLL will do nothing to any other process. Fortunately, we are able to identify our own applications' processes fairly easily. Also, we know that our applications never use the API function anywhere else, so this seems to be an acceptable solution. The fixup program is designed to run as a service under system account on NT-based Windows - code hooking requires administrator privileges. It works as a normal (hidden) Windows application on Windows 9x systems.

I wouldn't be able to do this without madCodeHook - madCollection is really a great library - in this case, a life saver. Thanks, Madshi, for your excellent work!

Tuesday, October 12, 2004

Delphi 2005 press release

Scotts Valley, Calif. - October 12, 2004: Borland Software (NASDAQ: NM: BORL) today announced Borland® Delphi™ 2005, previously codenamed "Diamondback" and the newest version of Borland's Rapid Application Development (RAD) environment for Windows® and .NET applications. Delphi 2005 combines Win32, .NET, Delphi and C# support all within one environment, significantly advances developer and team productivity and integrates with Borland's leading Application Lifecycle Management (ALM) solutions.

View the full press release here.

DataSnap.NET

About a week ago, I had an idea of porting the server part of DataSnap framework to .NET. Since then I've been fiddling with it in my free time, using Borland's C#Builder Personal Edition which is free for non-commercial development. Today, my first prototype of a (stateful) managed socket server is ready. So far, it's able to serve the following requests:
  • asGetAppServers: retrieve ProgIDs of installed appservers (thanks to Andrey Alifanov for this excellent article about using the COM Category Manager from .NET via interop)
  • asCreateObject: create instances of COM appservers
  • asFreeObject: release said instances
  • asInvoke: call simple methods, e.g. AS_GetProviderNames, AS_GetRecords (loading a client dataset works), as well as some simple custom methods of my test appserver.
An ASP.NET equivalent of httpsrvr.dll is also in the works (asynchronous HTTP handler, using the thread pool) but first I'd like to finish the common (transport-neutral) library which is responsible for marshalling DataSnap calls across wires via data packets.

I still have to debug and test marshalling of all possible parameter types (including multidimensional variant arrays, records, enums, byref parameters and so on). It seems to be easier to debug this code within the socket server which is a simple managed WinForms application.
A lot of my current code is, most probably, terribly inefficient, with lots of boxing and unboxing going on (first, data is read from the data packet stream into managed type variables which are then marshalled to unmanaged COM appservers, results unmarshalled to managed variables and streamed back to client). At this stage, I'm focusing purely on functionality; optimization can wait ;-).

What's the purpose of this work? Most of all, I'm doing it for my own fun and education. I'm still a beginner in .NET and this is a nice way for me to become more familiar with it. I have also acquired a bit deeper knowledge of this cool library called DataSnap.

Monday, September 27, 2004

Client dataset to .NET dataset

CC #21053: A great Delphi 8 project with source code written by Petr Vones. It can load a client dataset XML datapacket into a .NET dataset.

Sunday, September 26, 2004

Multicast events

I like multicast events in Microsoft .NET. Basically, a single event property (using Delphi terms) can be linked to multiple event handlers (as opposed to standard Delphi's events which can only be linked to a single event handler at most). In Delphi 7, a multicast technique is already used internally in TApplicationEvents, AppEvnts unit. In this case, the result is that you can have multiple TApplicationEvents components in your project (perhaps on different forms), each of which can receive and handle events coming from the global Application component.

It seems that some research is being done of multicast events support for future versions of Delphi for Win32.

In one of my projects, I've been experimenting with my own implementation (just for my own, non-visual classes, without any designtime support). It seems to be a success and I believe it helped me write cleaner, more modularized code. Writing it was fun, too.

Wednesday, September 22, 2004

Undocumented Delphi IDE command line switch

Another newsgroup nugget from Allen Bauer on b.p.d.n-t:

<quote>
One thing you can do with all versions of Delphi from at least D5, is use the "-r" command-line switch. This allows you to specify the root registry key to use when loading the IDE. For instance in D7, you could export the HKCU\Software\Borland\Delphi key to a file. Then rename the "Delphi" key to something else, like "SafeMode". Then re-import the exported key to recreate the HKCU\..\Delphi key. Finally, when you run the Delphi32 application, just pass the "-rSafeMode" switch to use that key.
</quote>

I think this feature can be very useful.

According to another post in that thread, there seem to be more undocumented switches: -s, -k, -q, -x, -y.
I have no idea what those are for...

Monday, September 20, 2004

Just a part of it, please (part 2)

For performance reasons, I have rewritten the ISAPI extension DLL (see my previous post) which simulates HTTP 1.1 GET with range specified.
The DLL now uses the IIS function HSE_REQ_TRANSMIT_FILE which is asynchronous and performance-wise optimized for sending files to clients.
Note: It helps to specify the FILE_FLAG_SEQUENTIAL_SCAN flag when opening the file. For asynchronous I/O operations, the FILE_FLAG_OVERLAPPED flag has to be specified.
For this purpose, I decided not to use Delphi's Web Application framework and wrote a plain ISAPI DLL instead.
Another modification that helped was not to use HTTP GET with parameters encoded directly in the URL. Instead, I now use HTTP POST with parameters encoded in binary POST data. DataSnap uses the same approach and it seems to be the most efficient one.
Unfortunately, our client applications are currently designed so that they send a lot of HTTP requests returning small packets of data. In such scenarios, it's better to specify HSE_IO_NODELAY flag in your HSE_REQ_TRANSMIT_FILE calls. Note: You can also use this flag with the WriteClient function (although it's not mentioned directly in its documentation).
This flag turns off the TCP nagling optimization (enabled by default) which, in case of small packets with Keep-alive connections, actually degrades performance because of the 200 ms timeout.
My subjective ;-) feeling is that the DLL's performance is now approximately the same as that of the standard HTTP 1.1 GET with range specified. I don't have enough statistical data yet to prove or disprove that speculation.

I have also finished a very simple ISAPI Filter DLL, whose only purpose is to be able to define some URL aliases. Again, some people have weird network configurations where their proxies forbid usage of URLs containing the .dll extension. The ISAPI filter modifies the incoming headers before IIS processes the request, so it can translate a URL like http://hostname/httpsrvr to http://hostname/scripts/httpsrvr.dll. Currently, it reads the alias definitions from the registry every time it's called by IIS, so the aliases are configurable without having to restart IIS. So far, this doesn't seem to adversely affect the performance.

CC #22334

Today I posted two performance counter libraries for instrumenting DataSnap appservers (one for sockets, one for HTTP transport) to CodeCentral.

Visual Poetry

Written in Delphi. Cool!

Saturday, September 18, 2004

CC #22332

Today I posted an example of creating a custom performance counter library to CodeCentral. I'm preparing my DataSnap performance libraries for publishing as well.

Saturday, September 04, 2004

YAPP

I'm using YAPP (Yet Another Pretty Printer ;-)) written by John Kaster to format my Delphi source code for HTML. The current version can highlight the following languages:
  • C++
  • C#
  • Delphi (of course!) ;-)
  • Java
  • SQL
  • Delphi forms (.dfm)
  • OMG IDL
  • Microsoft IDL
  • Xbase

It can produce highlighted output in HTML or RTF.
Quite useful. Thanks, John!

Just a part of it, please

A few days ago, I had to write an ISAPI extension DLL which would return partial content from a local file stored on the web server. This was meant as a workaround for some proxy configurations which (so I heard) strip some HTTP headers and thus make it impossible for our client code to simply use standard HTTP 1.1 GET requests with the Range header specified. (And before you ask, no, it was not acceptable to simply ask them to change their network configuration.)
So my ISAPI DLL should accept parameters like file name, start position and length in its URL and return only the requested bytes to the client (after translating the given file path to the actual local file path on the server).
The initial problem with Delphi's TWebResponse implementation (or so it seemed at first) was that it had no support for sending just a given number of bytes from a stream. The methods for sending response to the client are:


TWebResponse = class(TObject)
...
public
...
procedure SendResponse; virtual; abstract;
procedure SendRedirect(const URI: string); virtual; abstract;
procedure SendStream(AStream: TStream); virtual; abstract;
...
end;

The problem with TISAPIResponse.SendStream (TWebResponse is an abstract class, TISAPIResponse is the concrete descendant used in ISAPI applications) is that it will send the whole stream (or the rest of it from its current position) in 8K chunks until it reaches the end of the stream. So what are my options?

  1. Create a memory stream, copy the requested bytes to it and use it as the response content stream. Not a very good idea, if the requested range is large. This was my first, quick and dirty implementation, though.

  2. Write my own TWebResponse descendant and plug it into the framework - while perhaps possible, it might be a bit of work.

  3. Modify the VCL source code ;-) I don't like doing this unless for a very good reason.

  4. Anything else? Actually, yes.

My solution was to write a new TStream descendant:


type
TPartialFileStream = class(TStream)
private
FFileStream: TFileStream;
FSize: Int64;
FStartPos: Int64;
public
constructor Create(const FileName: string; AStartPos, ASize: Int64);
destructor Destroy; override;
function Read(var Buffer; Count: Longint): Longint; override;
function Write(const Buffer; Count: Longint): Longint; override;
function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; override;
end;

{ TPartialFileStream public }

constructor TPartialFileStream.Create(const FileName: string; AStartPos, ASize: Int64);
var
FileSize: Int64;
begin
inherited Create;
FFileStream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyNone);
FileSize := FFileStream.Size;
if (ASize < 0) or (ASize > FileSize) or (AStartPos < 0) or (AStartPos > FileSize) or
(FFileStream.Seek(AStartPos, soBeginning) <> AStartPos) then
raise EReadError.CreateRes(@SReadError);
FStartPos := AStartPos;
if ASize = 0 then
FSize := FileSize - FStartPos
else
FSize := ASize;
end;

destructor TPartialFileStream.Destroy;
begin
FFileStream.Free;
inherited Destroy;
end;

function TPartialFileStream.Read(var Buffer; Count: Longint): Longint;
begin
if FFileStream.Position + Count > FStartPos + FSize then
Count := FStartPos + FSize - FFileStream.Position;
if Count > 0 then
Result := FFileStream.Read(Buffer, Count)
else
Result := 0;
end;

function TPartialFileStream.Seek(const Offset: Int64; Origin: TSeekOrigin): Int64;
var
NewPos: Int64;
begin
case Origin of
soBeginning:
Result := FFileStream.Seek(FStartPos + Offset, soBeginning) - FStartPos;
soCurrent:
begin
NewPos := FFileStream.Position + Offset - FStartPos;
if NewPos < 0 then
NewPos := 0;
if NewPos > FSize then
NewPos := FSize;
Result := FFileStream.Seek(FStartPos + NewPos, soBeginning) - FStartPos;
end;
soEnd:
Result := FFileStream.Seek(FStartPos + FSize + Offset, soBeginning) - FStartPos;
else
Result := -1;
end;
end;

function TPartialFileStream.Write(const Buffer; Count: Longint): Longint;
begin
Result := 0; // this stream is read-only
end;

It uses TFileStream internally (and therefore accesses the file directly) but only "publishes" the specified part of the file. The code for sending the response is now very simple:


Stream := TPartialFileStream.Create(FileName, StartPos, Len);
Response.StatusCode := HTTP_STATUS_OK;
Response.ContentType := 'application/octet-stream';
Response.ContentStream := Stream;
Response.SendResponse;

Unicode Identifiers

News from Danny Thorpe about Unicode identifiers.

My personal and purely subjective note:
I just hope I will never have to read source code containing non-English (e.g. Chinese) identifiers.
I'm not a native English speaker but I shudder even when I think about source code containing identifiers in my own native language. Yuck! Am I too old already? ;-)

GExperts 1.2 released

Friday, September 03, 2004

Building packages from the command line

Allen Bauer has posted this little newsgroup nugget on b.p.d.n-t:

<quote>
The IDE supports a very little know feature where you can continue to control these options while in the IDE, yet allow the command-line be able to also control the options. In a package file, all the options listed there are propagated to all the contained units. This is different than a normal .dpr program/library file. In order to do this, all you need to do is replace the '$' with a ' ' (that's a <space>). Then when you open the dpk in the IDE, you can toggle those options as much as you like, but when you compile on the command-line, you can still control them.

This is how we build all our packages for the product itself. All of the Borland built packages have DEBUGINFO, STACKFRAMES, ASSERTIONS, OPTIMIZATION, LOCALSYMBOLS setup this way because like you, we wan't to be able to control those options from the command-line. The developers will typically do a complete "debug" build, whereas the integration team will not.

$DEFINES will sort of work the same way, in that if you replace the '$' with a ' ', the IDE will still recognize the defines and apply them, but any other modifications to the dpk source will re-insert the '$' character.
</quote>

Great to know. Thanks, Allen!

Thursday, September 02, 2004

PerfMon ready



My DataSnap appserver is now instrumented. In fact, my performance counter libraries are generic and will work with all DataSnap servers installed on the computer. DataSnap class IDs are read from the registry and a separate performance object instance is created per each class ID.
I have created two performance counter DLLs, one for web (httpsrvr.dll) and another for socket (scktsrvr.exe) transport.
I'm looking forward to watching the graphs later this month; some serious load is expected on our website when the users are back from their vacations.

Links to the source code:
Custom Performance Counters
Performance counter libraries for DataSnap appservers

Saturday, August 14, 2004

Friday, August 06, 2004

See the next Delphi release at BorCon 2004

In-depth details on DiamondBack, the code name for the next Delphi release, will be shown throughout BorCon 2004. If you use Delphi, you'll want to be there!
See the original BDN article by John Kaster.

Monday, August 02, 2004

First attempts in .NET

At home, I have a DSL connection which gets a different IP address every time I connect so I sometimes use the free dynamic DNS service. To update your registered DynDNS hostname with your current IP address, you need to send it a simple HTTP request conforming to their published specs.
I've been using a small tool I'd written to automate this task, using Delphi (of course!) and Indy with OpenSSL.
Today I tried to write a similar tool in C# and I was surprised how easy it was:


IPHostEntry hostInfo = Dns.Resolve(Dns.GetHostName());
if (hostInfo.AddressList.Length > 0)
{
string ipAddress = hostInfo.AddressList[0].ToString();

string url = "https://members.dyndns.org/nic/update" +
"?system=dyndns" +
"&hostname=" + userName + "." + hostName +
"&myip=" + ipAddress +
"&wildcard=OFF" +
"&mx=" +
"&backmx=NO" +
"&offline=NO";

HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
request.Credentials = new NetworkCredential(userName, password);
request.UserAgent = "tondrej dyndns test/1.0 tondrej@plonk.net";

HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Console.WriteLine(response.StatusCode);
}

SSL is used automagically just by using the "https://" prefix!

Just out of curiosity, I also tried it in Mono 1.0 on Linux (I use SuSE Linux 9.1). Unfortunately, the above code to retrieve my current IP address didn't work. Dns.Resolve() method seems to work differently in Mono: when I passed it my hostname it returned 127.0.0.2; when I passed it "localhost" it returned 127.0.0.1 (BTW, see this cool t-shirt).
I'll try to come up with code which works in Mono, however, this short experience showed me that it was probably naive to expect it to be fully compatible with Microsoft's .NET. For cross-platform code, you would probably get better results using just Mono on all target platforms (Mono runs on Windows, too).

Tuesday, July 27, 2004

CC #22009

Today I posted to Borland's CodeCentral an example which shows a possible way to access client IP address from a DataSnap remote data module.

Monday, July 19, 2004

My Delphi Tools

Here's list of tools I often use from my Delphi IDE (menu Tools\Configure Tools):

Title: Explore &Target Directory
Program: explorer.exe
Working Dir:
Parameters: /e,/select, $EXENAME

Title: Explore &Source Directory
Program: explorer.exe
Working Dir:
Parameters: /e,/select, $EDNAME

Title: Open Source in &Notepad2
Program: C:\Program Files\Notepad2\Notepad2.exe
Working Dir:
Parameters: $EDNAME

Title: &Console in Target Directory
Program: cmd.exe
Working Dir:
Parameters: /k cd $PATH($EXENAME)

Title: C&onsole in Source Directory
Program: cmd.exe
Working Dir:
Parameters: /k cd $PATH($EDNAME)

Title: Explore &Delphi Directory
Program: explorer.exe
Working Dir:
Parameters: "C:\Program Files\Borland\Delphi7",/e

Title: &Resource Explorer
Program: resxplor.exe
Working Dir:
Parameters: $EXENAME

Failure is rare

A quote from MSDN:


"There is no indication of success or failure. Failure is rare. There is no extended error information for this function; do not call GetLastError."


I think this gem should be archived for future generations ;-)
It's an excerpt from documentation of CharLower API function.

QC #8664

I've just posted this report to Borland's QualityCentral.
Under certain circumstances, httpsrvr.dll may make one additional, superfluous call to ReadClient after all data has already been read. In such case, the call will time out after 60 seconds.
Only happens if the client request content length is so big (from my tests, >48K) that it has to be read in chunks.

Friday, July 16, 2004

Localizing Borland runtime packages

Today, I saw someone on a Czech Delphi mailing list saying that it's not possible to localize Borland runtime packages using Delphi's built-in Translation Manager. Well, it certainly is possible; all you have to do is create a package project with the same structure (ie. containing the same units) as the target package.
The easiest way to create such a package is to let the Delphi IDE generate it for you ;-)
It does it when you double-click a .dcp node in the package designer, so, for example, if you want it to generate vcldb.bpl you first need to open a package which contains vcldb.dcp in its requires clause. Or create a new temporary .dpk project and add vcldb.dcp to its requires clause so you can double-click it.
Save the generated package project under a name corresponding with the target .bpl file name - for example, in Delphi 7 save generated vcldb.dpk as vcldb70.dpk. This will make the Translation Manager create the correct resource DLL project file name and you won't have to rename it before deployment.
From this point on, everything is pretty much standard - you use the Translation Manager to add new languages and translate the resources as usual. I've just tried it and it works. Easy, isn't it?

Tuesday, July 06, 2004

More silence

My shoulder needs surgery :-(
I'm going to the hospital tomorrow. Should be back home on Friday, if everything goes well. The right arm will be fixated again.
Oh my. More pain. More one-hand typing... Grrrrr.

Wednesday, June 30, 2004

From the Interbase trenches

Well, it seems that

select ... from my_proc(my_params, ...)
join my_table on ...


can be much slower than putting the code into a separate stored procedure and simply selecting * from it. Optimizer confused?
Oh well, I have to investigate some more, but some of my selects now run 6x faster... which is a good thing (TM) ;-)

Also, requesting plan on some complex queries (with a large plan) seems to crash ibserver. Doh... plan buffers limited in size to a hardcoded value? Found some hints on the newsgroups, turning off plan request on the client solves it, still it's hard to believe.

Saturday, June 26, 2004

Back

Long time no post... I had an accident on in-line skates, my right arm was dislocated from my shoulder. It hurts but it's getting better slowly, at least I can type with both hands, finally.

Thursday, May 20, 2004

Failure report ;-)

I had an idea to create a Delphi IDE add-in that would use existing IDE highlighters (hint: IOTAHighlighter interface declared in ToolsAPI unit) to convert source code currently open in the editor (or perhaps just selected lines) into an HTML page with highlighted syntax. The style/color codes could be read from the registry, thus respecting your current IDE settings.

Today I tried to do it but I failed miserably ;-)
It turns out that the IDE doesn't really expose its internal highlighters; you can get a reference to IOTAHighlighter interface but don't expect much from it. It seems to point to some generic implementation which does nothing but fill your buffer with $0E (SyntaxOff) codes. Doh ;-)

Perhaps I'm doing something wrong, but at the moment I cannot find out what that could be... In case you want to try it out and succeed where I have failed, don't hesitate to post your results to the Delphi opentools newsgroup ;-)

Sunday, May 16, 2004

initial commenting


Writeln('Hello, world!');

Welcome to my Delphi blog. Hope you enjoy it here.