[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: new version breaks some uses of unput()

>> On Mon, 14 Apr 2003 22:56:06 -0400,
>> Joey Hess <joeyh@debian.org> said: 

	Joey, I am going to be presumtuous and assume you would not
 mind this response going to a public list; I have condensed your
 questions down to the technical details, and you did imply that you
 considered filing a bug, so the email was not really personal and
 private. The reason is that there were a number of questions on IRC
 about similar topics. 

 > But before the code for the provided C functions is put in the C
 > file, flex now includes a line that undefs yytext_ptr.

	Yes, part of cleaning up after itself.

 > I can work around it easily enough, by either moving the functions
 > to the top code block, or by making them explicitly call yyunput
 > instead of the unput macro.

	Putting it in the top code block won't work, since the yyunput
 function is first implicitly declared, and then explicitly
 declared static. yyunput is not an published asset; so may go away at
 any time. In other words, there are macros and stuff available only
 in the rules section, for use in actions.

   The `flex' input file consists of three sections, separated by a
line containing only `%%'.
         user code

	Flex sets up a number of things that are available in the
 rules section, and now cleans it all up before polluting the user
 namespace. Now, actions can be any C statement, but using functions
 that in turn use flex macros is going to be a problem since declaring
 the function is going to be problematic.

	Some of these changes were driven by the need to make flex
 scanners reentrant, others by hte requirements for having multiple
 scanners, perhaps with different options, in the same program. 

 > Oh and why do I need an expliciy main() and yyweap() function now?
 > This code always worked before with flex generating stub functions
 > automatically. The real code is in the filters package, in
 > cockney.l, jive.l, and others.

	With multiple scanners now possible in the same program, flex
 would not know where to put main, obviously, it shoulkd not generate
 stubs in all the scanners generated.

	What is yyweap? 


/* code block */

/* Definitions Section */
BW [ \t\n]
SP [ \t]+
EW [ \t.,;!\?$]

  /* Rules Section */
foo  /* after regex */   {unput(yytext[yyleng-1]);}; /* after code block */
             /* Rules Section (indented) */
     /* User Code Section */
main(void) {
int yywrap() {
        return 1;

   An important potential problem when using `unput()' is that if you
 are using `%pointer' (the default), a call to `unput()' _destroys_
 the contents of `yytext', starting with its rightmost character and
 devouring one character to the left with each call.  If you need the
 value of `yytext' preserved after a call to `unput()' (as in the
 above example), you must either first copy it elsewhere, or build
 your scanner using `%array' instead. Also, `yytext' presently does
 _not_ dynamically grow if a call to `unput()' results in too much
 text being pushed back; instead, a run-time error results.

	This unput behaviour is not POSIX or lex compliant.  Also, The
 `unput()' routine is not redefinable. 

The bomb will never go off.  I speak as an expert in
explosives. Admiral William Leahy, U.S. Atomic Bomb Project
Manoj Srivastava   <srivasta@debian.org>  <http://www.debian.org/%7Esrivasta/>
1024R/C7261095 print CB D9 F4 12 68 07 E4 05  CC 2D 27 12 1D F5 E8 6E
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C

Reply to: