REBOL3 tracker
  0.9.12 beta
Ticket #0001915 User: anonymous

Project:



rss
TypeWish Statussubmitted Date15-Dec-2012 05:37
Version2.101.0 CategoryNative Submitted byBrianH
PlatformAll Severityminor Prioritynormal

Summary TRANSCODE /part option
Description Embedded scripts need to decode the binary source in a span from a start position to a particular end position, and shouldn't go any further. This kind of thing happens with the length-specified embedded scripts that sys/load-header enables, but it could also happen with something like RSP template dialects.

When we needed to support such limits in sys/load-header for checksums and decompression, we added a /part option to CHECKSUM and DECOMPRESS. We need a similar /part option for TRANSCODE. Preferably right after the source argument; because TRANSCODE is not called with APPLY at the moment it is safe to put it there, and that is the most useful position if it's called with APPLY in the future.

The /part option should take one argument of either the source series at the ending position or a length integer, similar to the behavior of /part elsewhere. TRANSCODE should treat that end position as the end of input, and trigger or insert any errors that it would normally do if it ran out of input at that point. If the /part length goes past the end of the source, that is not an error.

The return value would be the normal return value for the other options specified, but if normally for that source the continuation would be set to the end of source, if /part is specified it would be set to the end-of-part position instead. A user of TRANSCODE/part would be expected to have their code compare the position of the returned continuation to the end-of-part position, in cases where they would be checking for the tail position if they didn't specify /part. No special handling of the source-ends-before-part-length condition should be done; if that is a concern in your code, use end: skip source len to get the actual part ending position if you need something to check against.

Also see #1916 for the change needed to make this practical. The proposed spec below is based on the existing TRANSCODE spec, not any of the #1916 changed specs.
Example code
; Proposed spec of TRANSCODE with part option added:
transcode: native [
	"Translates UTF-8 binary source to values. Returns [value binary]."
	source [binary!] "Must be UTF-8 encoded"
	/part length [binary! integer!] "Length of source to decode"
	/next "Translate next complete value (blocks as single value)"
	/only "Translate only a single value (blocks dissected)"
	/error "Do not throw errors - return error object as value in place"
]

; For this example data:
>> s: to-binary {rebol [] print "Hello!" stuff after the script}
== #{
7265626F6C205B5D207072696E74202248656C6C6F2122207374756666206166
7465722074686520736372697074
}

; Code required now if you use to block!:
>> d: to block! copy/part s 23  ; note the copy overhead here
== [rebol [] print "Hello!"]
>> e: skip s 23
== #{2073747566662061667465722074686520736372697074}

; Code required now if you use transcode:
>> d: head clear back tail transcode copy/part s 23  ; still copy overhead
== [rebol [] print "Hello!"]
>> e: skip s 23
== #{2073747566662061667465722074686520736372697074}

; Yes, the clear back tail stuff is dumb, see #1916.

; Code required if you use the proposed transcode/part:
>> d: transcode/part s 23  ; no copy overhead here
== [rebol [] print "Hello!" #{2073747566662061667465722074686520736372697074}]
>> e: last d
== #{2073747566662061667465722074686520736372697074}
>> head clear back tail d
== [rebol [] print "Hello!"]

; Still has the stupid API clear back tail problem (see #1916), but at least it doesn't have the copy overhead anymore.

Assigned ton/a Fixed in- Last Update15-Dec-2012 21:52


Comments
(0003270)
BrianH
15-Dec-2012 21:48

Made the proposed behavior more explicit, and added a proposed spec and references to #1916.

Date User Field Action Change
15-Dec-2012 21:52 BrianH Description Modified -
15-Dec-2012 21:48 BrianH Comment : 0003270 Added -
15-Dec-2012 21:46 BrianH Description Modified -
15-Dec-2012 21:46 BrianH Code Modified -
15-Dec-2012 05:37 BrianH Ticket Added -