Welcome guest. Before posting on our computer help forum, you must register. Click here it's easy and free.

Author Topic: Scripting language embed in exe  (Read 8603 times)

0 Members and 1 Guest are viewing this topic.

Linux711

    Topic Starter


    Mentor

    Thanked: 59
    • Yes
    • Programming Blog
  • Certifications: List
  • Computer: Specs
  • Experience: Experienced
  • OS: Windows 7
Scripting language embed in exe
« on: June 15, 2011, 09:57:01 PM »
I started working on my scripting language project again, and decided that I want to be able to somehow embed my code with the interpreter and produce a single exe file that can be transferred to any machine without the interpreter. I figured that the way to do this would be using a resource file that contains the code which is read by the interpreter at run time. The problem is that I need to link the resource file. I just realized that the programming language I am using won't produce obj files, only exes.

So my questions are:

Is it possible to link a file that is already linked in order to add resources?

Is it possible to obtain an obj file from an exe?

Can I add my resources into the exe if it currently doesn't contain any resources? I know there is some winapi function called UpdateResource, but does that only replace already existing resources or can it add new ones?

* I have already determined that I don't want to just copy the code to the end of the exe because I want to be able to compress with UPX *

What should I do?  ???
YouTube

"Genius is persistence, not brain power." - Me

"Insomnia is just a byproduct of, "It can't be done"" - LaVolpe

BC_Programmer


    Mastermind
  • Typing is no substitute for thinking.
  • Thanked: 1140
    • Yes
    • Yes
    • BC-Programming.com
  • Certifications: List
  • Computer: Specs
  • Experience: Beginner
  • OS: Windows 11
Re: Scripting language embed in exe
« Reply #1 on: June 15, 2011, 10:06:27 PM »
I just realized that the programming language I am using won't produce obj files, only exes.
Which one?

Quote
Is it possible to link a file that is already linked in order to add resources?
I don't think so. obj files don't have resources, anyway- those are added by a resource compiler after the fact.
Quote
Can I add my resources into the exe if it currently doesn't contain any resources? I know there is some winapi function called UpdateResource, but does that only replace already existing resources or can it add new ones?
It does both. ref:

http://msdn.microsoft.com/en-us/library/ff468902%28v=VS.85%29.aspx

Quote
because I want to be able to compress with UPX
nice. Isn't it awesome being flagged as a trojan by all sorts of AV programs?


I was trying to dereference Null Pointers before it was cool.

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Scripting language embed in exe
« Reply #2 on: June 16, 2011, 12:09:07 AM »
Here I offer  a very broad reply,  void of specific details.  Interpretive language design is a very broad thing.

They most widely used interpretive language presently used and widely known, is Java. here are others, but not widely know outside of specific industries. Hard to say which is more important. You just don't see a big business with a poster reading  "Jave Inside" on the front windows. The poster  could say "MS SQL used here' just as well.

Using  the model of Java or some of the others, you can create your own interpretive language that is cross-platform.

But to include the run-time library and the pseudo-code in one package is not using the design to its full potential. The resulting EXE files are larger that needed. You would wind up with EXE files larger than the run-time package.

Historically, this has been done many times.The technique is still used in commercial applications. Especially large data base management systems. An interpretive  code is need both for debugging and maintainability of the massive programs used in large systems. They write books about this. Boring books.

You might want to learn more about the MS SQL.
http://en.wikipedia.org/wiki/Microsoft_SQL_server
« Last Edit: June 16, 2011, 12:24:07 AM by Geek-9pm »

BC_Programmer


    Mastermind
  • Typing is no substitute for thinking.
  • Thanked: 1140
    • Yes
    • Yes
    • BC-Programming.com
  • Certifications: List
  • Computer: Specs
  • Experience: Beginner
  • OS: Windows 11
Re: Scripting language embed in exe
« Reply #3 on: June 16, 2011, 12:45:43 AM »
They most widely used interpretive language presently used and widely known, is Java. here are others, but not widely know outside of specific industries. Hard to say which is more important. You just don't see a big business with a poster reading  "Jave Inside" on the front windows. The poster  could say "MS SQL used here' just as well.

Java is not an interpreted language. The interpreted language is Java bytecode, which is quite a bit different. java isn't the only language compiled to bytecode; two examples I can name of the top of my head that also compile to the Java Virtual Machine are Erlang and Scala.



Quote
Using  the model of Java or some of the others, you can create your own interpretive language that is cross-platform.
He already has an interpreted language. He wants to wrap it in an executable form; that is, interpreter + interpreted script in one go. He's not trying to make a virtual machine.

Quote
But to include the run-time library and the pseudo-code in one package is not using the design to its full potential.
What does this have to do with anything?

Quote
The resulting EXE files are larger that needed. You would wind up with EXE files larger than the run-time package.
So you are instead suggesting that he get people to go and download his interpreter? That is exactly the reason he wants to make it this way, so they don't have to take that extra step. Even now, to run Java applications, one needs to install a JVM; to run .NET applications, you need a CLR (although one comes with Vista & 7). His "runtime" (script language interpreter) is not going to be super popular or familiar to people, so it isn't something they are going to have installed.

Quote
Historically, this has been done many times.The technique is still used in commercial applications. Especially large data base management systems.
What are you even talking about now? embedding script code and stored procedures in a database is completely and entirely different from packaging a Interpreter with the interpreted script, and have completely different use cases.

Quote
An interpretive  code is need both for debugging and maintainability of the massive programs used in large systems. They write books about this. Boring books.

You might want to learn more about the MS SQL.
http://en.wikipedia.org/wiki/Microsoft_SQL_server

Somebody asks how to build a tool similar to py2exe for his scripting language implementation.. and you provide a link to the SQL Server Wiki page.

I... I cannot fathom what logic hides behind this response.

Either way, I just had a thought about the UPX compression/encryption:

is it being compressed <after> you modify the resources? because you cannot use the Resource API to fiddle with the resources of a program compressed with most compression formats. (UPX I know is one of them).

Best thing to do would be to have the script interpreter program check when it starts for a given resource, and if not found, proceed to act "normally", but if it is, it uses that resource as the script to execute. Then when you want to create a "self-running" executable, you simply make a copy of the interpreter program, and then use the Resource API to add the resource and make it the script you want to execute.
I was trying to dereference Null Pointers before it was cool.

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Scripting language embed in exe
« Reply #4 on: June 16, 2011, 02:45:56 AM »
Your dedicatory skills really suffers BC.

1,330 Java experts say it is not an interpreted language.
39,400 Other authorities say is is in deed and interpretive language. Go with the majority, unless you are one of those experts.

To carry this further, some say Forth is an inoperative language, yet others argue that it is not even a language, just a way for invoking low-level code. Some real-time industrial applications are structures like any as interpretive language, yet they are very low-level primitive systems just a cut above machine code.

And yes, virtual  machine, at the core concept, are interpretative  systems.

Quote
Many interpreted languages are first compiled to some form of virtual machine code, which is then either interpreted or compiled at run time to native code.

If he wants to incorporate his interpreter and his run-time library in on bundle, yes he can do it. If he wants to optimize, it can be made to take very little space and can be a self-extracting file. And he does not have to use .NET or any of that kind of trash. Just a lot of work.

He can use an Natural, Organic, pesticide-free, Green, low salt no fat c/C++ compiler from some one other that MS. Did I hear Intel?
http://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler

Linux711

    Topic Starter


    Mentor

    Thanked: 59
    • Yes
    • Programming Blog
  • Certifications: List
  • Computer: Specs
  • Experience: Experienced
  • OS: Windows 7
Re: Scripting language embed in exe
« Reply #5 on: June 16, 2011, 03:03:30 AM »
Quote
I just realized that the programming language I am using won't produce obj files, only exes.
Quote
Which one?

PB (PowerBasic). Originally I started my project in C, but I stopped half way because it became too cumbersome, and I don't like C syntax. I thought about doing it in VB6, but it's getting old and uses a runtime. I didn't like the idea of making an interpreter in such a high-level language, so I decided on something in between - PowerBasic. The syntax is almost identical to VB, but it compiles to a native exe. So far, I have been liking it, except for two things. It won't produce obj files and it uses a proprietary resource file format (read below).

Quote
nice. Isn't it awesome being flagged as a trojan by all sorts of AV programs?

Are you talking about UPX? If so, can you recommend a exe compressor/encryptor that isn't detected by AV.

Quote
Historically, this has been done many times.The technique is still used in commercial applications. Especially large data base management systems. An interpretive  code is need both for debugging and maintainability of the massive programs used in large systems.

@GEEK: Are you tired?  :)

Quote
Best thing to do would be to have the script interpreter program check when it starts for a given resource, and if not found, proceed to act "normally", but if it is, it uses that resource as the script to execute. Then when you want to create a "self-running" executable, you simply make a copy of the interpreter program, and then use the Resource API to add the resource and make it the script you want to execute.

That's what I was going to do until I experimented with ResourceHacker and found out that PB uses a different kind of resource file format, so this method won't work. I think I am just going to write the code directly on to the end of the exe file and hope it doesn't trigger any AV software.

Quote
If he wants to incorporate his interpreter and his run-time library in on bundle, yes he can do it. If he wants to optimize, it can be made to take very little space and can be a self-extracting file. And he does not have to use .NET or any of that kind of trash. Just a lot of work.

He can use an Natural, Organic, pesticide-free, Green, low salt no fat c/C++ compiler from some one other that MS. Did I hear Intel?

Yes, that's what I plan on doing. I don't think the size of the runtime will matter because it's only about 100KB right now and it will probably not go above 300KB which is really small compared to most runtimes. As for the suggestion for a compiler, I am already working on the interpreter and don't need to change languages now (I already did once).
YouTube

"Genius is persistence, not brain power." - Me

"Insomnia is just a byproduct of, "It can't be done"" - LaVolpe

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Scripting language embed in exe
« Reply #6 on: June 16, 2011, 03:39:21 AM »
OK. You are serious. Forget using C.
With Power Basic you can don it all. With just a little bit of extra code to 'glue' things together.
Once you get on to it, please don't tell how it is done.  Some of us make our living keeping these kinds of things secret. You might know what I mean. Use C when you work for a big company. When doing work  privately, use  secret tools. I got that down about 15 years ago on a critical project that started d a small company. Doing it is C was a big was of time. even with the best C libraries we could get back thing. There were three or four secret tools I had. One was called Framework II. Too bad MS bought it and froze it.

BC_Programmer


    Mastermind
  • Typing is no substitute for thinking.
  • Thanked: 1140
    • Yes
    • Yes
    • BC-Programming.com
  • Certifications: List
  • Computer: Specs
  • Experience: Beginner
  • OS: Windows 11
Re: Scripting language embed in exe
« Reply #7 on: June 16, 2011, 03:46:54 AM »
Your dedicatory skills really suffers BC.
my what?
Quote
1,330 Java experts say it is not an interpreted language.
39,400 Other authorities say is is in deed and interpretive language.
*censored*? you can't just plug search queries into google and use those results as some sort of evidence. Nor can you claim that the hits are text being posted by "java experts" so you can stop making stuff up now.

Java itself isn't an "interpreted" language, the bytecode is. That may sound like "no difference" but there is a difference. Python/Perl/VBScript/Javascript have to be read, parsed, interpreted and executed On-the-fly; java is compiled to bytecode, and that bytecode is compiled to native machine code as the program runs. is there an interpretation step? Of course there is. But my point is that the way java is run using a Virtual Machine is completely and utterly different from a standard script language and is not something that you create for a portable script language. To run java code, it needs to be compiled. After that point- it's no longer java code- it's bytecode. Therefore java is a compiled language. Perhaps not in the spirit of the definition but that's the way the cards stacked.

Quote
To carry this further, some say Forth is an inoperative language, yet others argue that it is not even a language, just a way for invoking low-level code. Some real-time industrial applications are structures like any as interpretive language, yet they are very low-level primitive systems just a cut above machine code.
What are you talking about? Where the *censored* did Forth come from here?
Quote
And yes, virtual  machine, at the core concept, are interpretative  systems.
But they are still fundamentally different from Perl, Python, VBScript, etc. And regardless is NOT WHAT THE OP HAS. They aren't asking "how do I implement a Interpreted language". their question wasn't, "hey, I already have a language interpreter I wrote, can somebody make a nonsense post about java and virtual machines that does nothing to answer my actual question regarding the creation of an Executable package for a script file?"

Quote
If he wants to incorporate his interpreter and his run-time library in on bundle, yes he can do it.
Well thank goodness you are here to tell us all what we already know.


Quote
If he wants to optimize, it can be made to take very little space and can be a self-extracting file.
What are you even talking about now. I truly do not know. He never said he wanted to optimize. How a "self-extracting file" would help is beyond me. I honest have no clue what you are rambling about.



Quote
And he does not have to use .NET or any of that kind of trash.

.NET is not trash. You ought to back up your statements with some sort of evidence. Ideally not "evidence" regarding the number of hits you get between ".NET is trash" and ".NET is not trash". Remember to back it up with statements about .NET (the class library) and not the CLR (the runtime); I assume you of course are well aware of the difference, being that you are able to make such blanket statements about the technologies as a whole.

points to pay close attention to:

Why is it trash? Again: Specifics. none of this "well it does <insert vague statement> or it doesn't allow me to <insert generalized, handwavey and probably wrong statement>" either.
How would using .NET make this "task" easier? I really don't see how it would. Messing about with embedded resx files would be a *censored* of a lot more difficult than working with the Resource API functions. The only resource a .NET application would normally have are things like icon and version; and possibly typelib if it is a COM wrapper, but I haven't used COM wrappers. needless to say I'm sure you regard COM as trash as well. *censored* anything you don't understand in the slightest must be trash.

Quote
He can use an Natural, Organic, pesticide-free, Green, low salt no fat c/C++ compiler from some one other that MS. Did I hear Intel?
http://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler

This has absolutely NOTHING TO DO WITH ANYTHING IN THIS THREAD. But I'll address it.

What is different between Intel's Compiler and Microsoft's compiler?

Specifics please. Pay particular attention to those elements that would make it more natural, Organic, pesticide free, Green, and no fat. Or, perhaps to be precise, why they shouldn't use the MS compiler in favour of it. And also, why the Microsoft C/C++ compiler even enters into this thread since they didn't state what language they used. Why Intel, anyway? Most Anti-Microsoft peeps prefer to use gcc or g++ for C++ (and to be fair it is a fine compiler).


Quote
It won't produce obj files and it uses a proprietary resource file format (read below).
Actually, all windows executables will have a standard resource- you can use the Resource API to add a new entry.

however, the difference with PowerBASIC in  this case would be that you need to read that Resource using the API as well. (rather than using the built in "resource handling" that it provides. LoadString() might be sufficient for attempting to read the appropriate resource. from the code; adding the resource ought to still be a appropriate set of Resource management functions to update the resource.


Quote
Are you talking about UPX? If so, can you recommend a exe compressor/encryptor that isn't detected by AV.
None that I know of. FWIW I personally find exe compression/encryption utterly pointless and only goes to show ones hubris about their own projects. And this is with PowerBASIC, which if I understand correctly compiles to a Machine code EXE, right? isn't that enough "protection" from... whatever you are encrypting the program for? After all, only you have the original source, anybody else would just have the dissassembly.

I was trying to dereference Null Pointers before it was cool.

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Scripting language embed in exe
« Reply #8 on: June 16, 2011, 11:14:03 AM »
Quote
I personally find exe compression/encryption utterly pointless and only goes to show ones hubris about their own projects
We can agree on this. Compression/encryption is of little value for a project. Unless compression/encryption is the project.

From the OP first post it is not clear if he has and interactive language or one the has to be processed before execution. Many compiled languages have other forms, even interactive interpreters.

You keep using the term 'byte code'. Would you like to explain that? Some machines have a 12 bit word.

BC_Programmer


    Mastermind
  • Typing is no substitute for thinking.
  • Thanked: 1140
    • Yes
    • Yes
    • BC-Programming.com
  • Certifications: List
  • Computer: Specs
  • Experience: Beginner
  • OS: Windows 11
Re: Scripting language embed in exe
« Reply #9 on: June 16, 2011, 02:38:44 PM »

You keep using the term 'byte code'. Would you like to explain that? Some machines have a 12 bit word.

http://en.wikipedia.org/wiki/Java_bytecode
I was trying to dereference Null Pointers before it was cool.

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Scripting language embed in exe
« Reply #10 on: June 16, 2011, 03:10:50 PM »
http://en.wikipedia.org/wiki/Java_bytecode
Thank you. That is informative and contains material that not all readers already know. Such as:
Quote
Java bytecode is the form of instructions that the Java virtual machine executes. Each bytecode opcode is one byte in length, although some require parameters, resulting in some multi-byte instructions. Not all of the possible 256 opcodes are used. In fact, Sun Microsystems, the original creators of the Java programming language, the Java virtual machine and other components of the Java Runtime Environment (JRE), have set aside 3 values to be permanently unimplemented.[1
...
A Java programmer does not need to be aware of or understand Java bytecode at all. However, as suggested in the IBM developerWorks journal, "Understanding bytecode and what bytecode is likely to be generated by a Java compiler helps the Java programmer in the same way that knowledge of assembler helps the C or C++ programmer."[2]
...
Java bytecode is designed to be executed in a Java virtual machine. There are several virtual machines available today, both free and commercial products.
  Further information: Java virtual machine
If executing Java bytecode in a Java virtual machine is not desirable, a developer can also compile Java source code or Java bytecode directly to native machine code with tools such as the GNU Compiler for Java. Some processors can execute Java bytecode natively. Such processors are known as Java processors.
Where did the OP go? Having a cup of coffee?

Salmon Trout

  • Guest
Re: Scripting language embed in exe
« Reply #11 on: June 16, 2011, 03:17:13 PM »
Your dedicatory skills really suffers BC.

What does this mean?

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Scripting language embed in exe
« Reply #12 on: June 16, 2011, 08:47:49 PM »
What does this mean?
Intended here as a form of dry sarcasm.

It refers to the skill of raising or elevating a literary or musical work to a higher level of unselfish inspiration by dedicating the  work to another person or Deity or esteemed principal.  Sometimes used by authors as a shield to hide pride and arrogance. "My amazing  hyper-drive theory book is dedicated to the memory my dear sweet aunt Martha, who used to instill in me the desire to reach for the stats."

Linux711

    Topic Starter


    Mentor

    Thanked: 59
    • Yes
    • Programming Blog
  • Certifications: List
  • Computer: Specs
  • Experience: Experienced
  • OS: Windows 7
Re: Scripting language embed in exe
« Reply #13 on: June 16, 2011, 09:43:06 PM »
Quote
which if I understand correctly compiles to a Machine code EXE, right? isn't that enough "protection" from... whatever you are encrypting the program for?

The reason I want to encrypt it, is not to protect from disassembling the interpreter, but from reading the script contained within the exe. In all other instances though, yes, I do believe that encrypters/compressors are not necessary.

Quote
We can agree on this. Compression/encryption is of little value for a project. Unless compression/encryption is the project.

Or if there is a script inside the exe.

Quote
From the OP first post it is not clear if he has and interactive language or one the has to be processed before execution.

No, there is no processing before execution in my language, aside from tokenizing the syntax. It's pretty simple. Just a major switch statement to process the commands and some modules which contain the functions. Currently, it has no debugging or error handling. I don't want to spend too much time writing all the error messages that it would require. This is not something that I am doing for business, just a hobby.

The language is like a mixture of BASIC and batch. The idea behind it is to make a language that is like batch, but also has access to API functions. Like AutoIT.

Quote
Where did the OP go? Having a cup of coffee?

Sleep.

Just incase you missed this earlier:
Quote
That's what I was going to do until I experimented with ResourceHacker and found out that PB uses a different kind of resource file format, so this method won't work. I think I am just going to write the code directly on to the end of the exe file and hope it doesn't trigger any AV software.

I used resource hacker and also tried the resource API functions that you posted earlier. No go, they both corrupt the exe. So that's how I arrived at the conclusion that PowerBASIC uses a different resource format. I could be wrong, but I think investigating that would be hard.
YouTube

"Genius is persistence, not brain power." - Me

"Insomnia is just a byproduct of, "It can't be done"" - LaVolpe

Linux711

    Topic Starter


    Mentor

    Thanked: 59
    • Yes
    • Programming Blog
  • Certifications: List
  • Computer: Specs
  • Experience: Experienced
  • OS: Windows 7
Re: Scripting language embed in exe
« Reply #14 on: June 16, 2011, 10:01:01 PM »
Incase you are interested, here is my first test script:

Code: [Select]
define:value: count
define:value: i
define:text: thing

SetValue count 5

Out Type something: $
In thing

->Repeat [value] count
     + i
     Out You typed: [text] thing [value] i $
<-

Output:
Code: [Select]
Type something: bla bla
You typed:bla bla1
You typed:bla bla2
You typed:bla bla3
You typed:bla bla4
You typed:bla bla5
YouTube

"Genius is persistence, not brain power." - Me

"Insomnia is just a byproduct of, "It can't be done"" - LaVolpe