Error in ReadHTMLFile
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
dseligo · 2023-11-15 21:44

ReadHTMLFile procedure (from SAX_HTML unit) throws exception when
reading this HTML text:
Code: HTML5  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i <!DOCTYPE
html><html lang="en"><head><title>Test</title></head><body><div
title="test<"></div></body></html>
https://validator.w3.org says that above HTML is without errors or
warnings.

Should I report a bug?

Test program:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
htmlproblem; {$mode objfpc}{$H+} uses SysUtils, Classes, DOM,
DOM_HTML, SAX_HTML; var  sHTML: String =        '<!DOCTYPE html><html
lang="en"><head><title>Test</title></head><body>' +        '<div
title="test<"></div>' +        '</body></html>';  HTMLDocument:
THTMLDocument;  iPos: Integer; procedure ReadHTML(AHTML:
String);begin  WriteLn('Reading: ', AHTML);  try    try     
ReadHTMLFile(HTMLDocument, TStringStream.Create(AHTML));     
WriteLn('Read OK.');    except      on E: Exception do      begin    
  WriteLn('Read not OK: ', E.Message);        If E is EDOMError then 
       WriteLn('EDOMError code: ', (E as EDOMError).Code);      end; 
 end;    WriteLn;  finally    HTMLDocument.Free;  end;end; begin 
ReadHTML(sHTML);   // remove less-than sign  iPos := Pos('<">',
sHTML);  Delete(sHTML, iPos, 1);   ReadHTML(sHTML);end.

----------------------------------------------------------------------
marcov · 2023-11-15 21:50

Usually < and > and & need to be escaped.

----------------------------------------------------------------------
rvk · 2023-11-15 21:58

Mentioning the exact exception would be helpful too

Technically the < is allowed in the values of an attribute.
But it is strongly discouraged and advised you encode those characters
(like in normal text).

But yes, it is allowed so if that's the reason for the exception you
could technically consider it a bug.

----------------------------------------------------------------------
dseligo · 2023-11-15 22:56

Quote from: rvk on November 15, 2023, 10:58:46 pmMentioning the exact
exception would be helpful too

Exception is 'EDOMError in DOMDocument.CreateElement', code is 5
(INVALID_CHARACTER_ERR).

QuoteBut yes, it is allowed so if that's the reason for the exception
you could technically consider it a bug.

I created bug report:
https://gitlab.com/freepascal.org/fpc/source/-/issues/40523

I forgot to mention compiler version: Free Pascal Compiler version
3.3.1-14158-g6fda6f79d8 [2023/10/23] for x86_64.

----------------------------------------------------------------------
dseligo · 2023-11-15 22:59

Quote from: marcov on November 15, 2023, 10:50:09 pmUsually < and >
and & need to be escaped.

I have no influence on HTML in question, but (as rvk also said) less-
than sign doesn't need to be escaped if value of attribute is quoted.

----------------------------------------------------------------------
lainz · 2023-11-15 23:54

< sign is usually badly parsed even by vs code, even if quoted, the
highlighter simply doesn't like it and threat it as part of html
element...

Attached screenshot. Char in red

----------------------------------------------------------------------
marcov · 2023-11-16 08:25

I'll have a look in the weekend, but a good chance that the same
function is used for all literal string reading in some tokenizing
stage.  Might not be easy.

----------------------------------------------------------------------
dseligo · 2023-11-16 10:08

Quote from: marcov on November 16, 2023, 09:25:28 amI'll have a look
in the weekend, but a good chance that the same function is used for
all literal string reading in some tokenizing stage.  Might not be
easy.

IMHO, problem is check in the line 296 of method THTMLReader.Parse in
unit SAX_HTML. It enters 'scUknown' context after that, while it
should stay in the same token.

There is already this comment there:
Code: Text  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i           
    // TODO: this check is hardly complete, probably must also check
if                // tag name is followed by legal attributes.