Re: new version breaks some uses of unput()
Hi,
>> On Mon, 14 Apr 2003 22:56:06 -0400,
>> Joey Hess <joeyh@debian.org> said:
Joey, I am going to be presumtuous and assume you would not
mind this response going to a public list; I have condensed your
questions down to the technical details, and you did imply that you
considered filing a bug, so the email was not really personal and
private. The reason is that there were a number of questions on IRC
about similar topics.
> But before the code for the provided C functions is put in the C
> file, flex now includes a line that undefs yytext_ptr.
Yes, part of cleaning up after itself.
> I can work around it easily enough, by either moving the functions
> to the top code block, or by making them explicitly call yyunput
> instead of the unput macro.
Putting it in the top code block won't work, since the yyunput
function is first implicitly declared, and then explicitly
declared static. yyunput is not an published asset; so may go away at
any time. In other words, there are macros and stuff available only
in the rules section, for use in actions.
The `flex' input file consists of three sections, separated by a
line containing only `%%'.
definitions
%%
rules
%%
user code
Flex sets up a number of things that are available in the
rules section, and now cleans it all up before polluting the user
namespace. Now, actions can be any C statement, but using functions
that in turn use flex macros is going to be a problem since declaring
the function is going to be problematic.
Some of these changes were driven by the need to make flex
scanners reentrant, others by hte requirements for having multiple
scanners, perhaps with different options, in the same program.
> Oh and why do I need an expliciy main() and yyweap() function now?
> This code always worked before with flex generating stub functions
> automatically. The real code is in the filters package, in
> cockney.l, jive.l, and others.
With multiple scanners now possible in the same program, flex
would not know where to put main, obviously, it shoulkd not generate
stubs in all the scanners generated.
What is yyweap?
manoj
%{
/* code block */
%}
/* Definitions Section */
BW [ \t\n]
SP [ \t]+
EW [ \t.,;!\?$]
%%
/* Rules Section */
foo /* after regex */ {unput(yytext[yyleng-1]);}; /* after code block */
/* Rules Section (indented) */
%%
/* User Code Section */
int
main(void) {
yylex();
}
int yywrap() {
return 1;
}
ps.
An important potential problem when using `unput()' is that if you
are using `%pointer' (the default), a call to `unput()' _destroys_
the contents of `yytext', starting with its rightmost character and
devouring one character to the left with each call. If you need the
value of `yytext' preserved after a call to `unput()' (as in the
above example), you must either first copy it elsewhere, or build
your scanner using `%array' instead. Also, `yytext' presently does
_not_ dynamically grow if a call to `unput()' results in too much
text being pushed back; instead, a run-time error results.
This unput behaviour is not POSIX or lex compliant. Also, The
`unput()' routine is not redefinable.
--
The bomb will never go off. I speak as an expert in
explosives. Admiral William Leahy, U.S. Atomic Bomb Project
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/%7Esrivasta/>
1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
Reply to: