×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

stack getting clobbered (value of esp)

stack getting clobbered (value of esp)

stack getting clobbered (value of esp)

(OP)
i wrote a recursive function to search a tree structure, and i'm having problems with the code.  the value of esp is not getting preserved over the recursive calls, but the strange thing is that as far as i can tell, upon every function entrance and corresponding function exit, the value is the same (i.e., i'm popping as much as i'm pushing, etc.).

is there something i'm forgetting about calling or returning from functions?  are there any other things having to do with esp that occur when calling/returning from functions besides a push/pop of the caller's ip?

RE: stack getting clobbered (value of esp)

You'll have to post the code, but here's some for starters:
(1) If the routine is recursive, then esp will be different on each call, because each call will go deeper and deeper into the stack.
(2) The size of the address you have on the stack depends on whether you are calling far or near, and on 32-bit or 16-bit offset.
(3) Yes, functions often do more than just push the address. If you want local variables then you will normally push bp (ebp) and set ebp to esp so as to address the local variables relative to ebp. If you passed parameters, this can be done by stack too. But if you're in assembler you know what you're passing and how...

RE: stack getting clobbered (value of esp)

Lionelhill's probably right: you'd probably best post your code for quickest response.

--------
It is an honor to recieve an honest reply. (Prov 24:26)

RE: stack getting clobbered (value of esp)

(OP)
i would have posted the code in the first place but it is kind of long.

obviously there is something i'm not doing right, i just can't tell what it is.  but as long as i know that nothing else *should* be happening to the stack, then i know it's a bug in my code and i can eventually find it, as opposed to it something happening i didnt realize (i.e. somethning that occurs with ebp, esp, recursive functions, etc.).

RE: stack getting clobbered (value of esp)

(OP)
something very strange when i'm debugging with msvc++ 6...

if i step into every function call, esp doesn't get screwed up.  however, if i step over a function call, it gets changed.

to be more detailed, there are two sections of the recursive function, one to handle leaf nodes and one to handle parent nodes.  there is a jump that moves to different parts of the code depending on parent/leaf status.  i can step over calls that would result in a leaf node being processed with esp remaining the same, but if i step over the a call that would process a parent, esp is wasted.  in fact, it gets set to the what esp was when the last leaf node returned (that is, what esp was at the end of the recursion).  mind you, this only happens if i step over parent node calls.  if i step into and through it, esp is OK.

basically i can't tell why esp is getting clobbered, since it's fine if i step through it with the debugger.

RE: stack getting clobbered (value of esp)

I'm deeply suspicious about this, because things nearly always run with the debugger exactly as they would without. But there is one exception (probably more! There's one exception I know of!)
If you (ill-advisedly!) do a conditional jump by rewriting the next instruction (i.e. self-modifying code) it will work find in a debugger but probably not in "real life", where the instruction may already be in the prefetch queue before it got modified.
But I very much doubt this is your problem.

RE: stack getting clobbered (value of esp)

....demon posession?

--------
It is an honor to recieve an honest reply. (Prov 24:26)

RE: stack getting clobbered (value of esp)

defenitely we need a least the proc entrance/exit code to look at for making conclusions. little comment to lionehill - as far as i know any jump (in fact, any instruction changing IP) invalidates the prefetch queue. at least the things were this way for i486

regards

RE: stack getting clobbered (value of esp)

oleksii, you're probably right. I'm remembering reading something a long time ago, possibly pre 486, in a dubious chapter on hints how to make your program hard to disassemble (a fairly futile thing to do anyway. How many of us will ever write something that anyone would actually want to disassemble?)

RE: stack getting clobbered (value of esp)

to lionelhill. probably u'll code a crackme one day? :)
the stuff u mentioned probably concerns jumps to a "misaligned" instruction - there is really such a antidebugging trick. concerning the prefetch queue - it's absolutely transparent on the coder side, the only thing one can bother is speed optimization: less jumps, faster execution, at least in theory..

i wonder if jorgander has found the answer? :)

RE: stack getting clobbered (value of esp)

Thanks for info...

RE: stack getting clobbered (value of esp)

(OP)
sorry i haven't been on this forum in a while...

yes, i solved the problem, and sadly it was my own naivette.  u see, i am used to debugging non-assembler code with msvc++, and in the debugger when u are running a recursive function and u "step over" a function call, the debugger will stop again when ***that particular instance*** of the recursive function returns.  in assembler code, however, where the debugger does not have access to the high-level code, it cannot resolve complicated things like remembering which instance of a recursive function call to stop at, etc. (of course, i'm assuming these things about msvc++'s debugger here).

so, since i was debugging a recursive function that was written to search a tree structure, the debugger was stopping at the ***most recent instance*** of the instruction, not excluding those times the instruction was called in other instances of the same function.

in short, i was assuming that the debugger was stopping at  the same level in the tree hierarchy, when in fact it was stopping much farther down (at the bottom, in fact).  *that* is why i thought esp was not getting preserved over function calls.

RE: stack getting clobbered (value of esp)

Thanks for telling us the answer. It is always good to remember things like that ready for when it happens to me, too.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login


Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close