BK Midi Importer

[runehero123]

[UPDATE!]
Version 1.02 is out now, grab it here:

http://www.mediafire.com/download.php?ic4c4tngvov

Hey everyone,

For the past couple months I have been working on parsing the BK Midi files, and today I present the first ever imported Midi files to Banjo-Kazooie. The video contains three separate songs which were converted from a Midi file to the BK music format using my Midi Converter application.

http://www.youtube.com/watch?v=MbU_OqpHMBA

If you notice, the second song has a "hanging" note(doesn't turn off for some reason), and I have no idea what is causing that. Anyways, I'm sure you're wondering how the application will work. All it takes is four steps to convert your midi.

1 - Import your midi file(Type 1 only).
2 - Select an instrument for each track
3 - Export your midi file to the BK music format (compressed midi)
4 - Inject the midi into your BK Rom file at another song's address.

That's pretty much it. You will probably have to do some trial and error when selecting instruments, as not all of the instruments sound great when you first import the song. It probably took me about 5 revisions for each song, to get it sounding okay.

Support for DK64 and BK will be added later. Thanks to Coolboyman for helping with debugging.

[Edit]
Song files in the video have been included in the attachments.

[SubDrag]

That is fantastic!!! I also really hope i can apply it to GE too.

[runehero123]

That is fantastic!!! I also really hope i can apply it to GE too.

Thanks :) . I've heard people say GE used the N64 compressed midi format, so it should be exactly the same or very similar to BK's songs. I'll have to check one of the song files out myself to be sure.

[Ice Mario]

Nice work. Some kind of soundbank for the most popular sequencers that includes all the BK instruments would be a nice addition, so they get to know what they sound like before importing, and then when you import the midi you can automatically get the correct indices for the tracks.

Again, good work.

[runehero123]

Nice work. Some kind of soundbank for the most popular sequencers that includes all the BK instruments would be a nice addition, so they get to know what they sound like before importing, and then when you import the midi you can automatically get the correct indices for the tracks.

Good idea. I'll definitely look into that as soon as I release the first version.

[Ccd-ToF]

Great projet and good job ! :)

[runehero123]

If someone can find/provide a midi for any Nuts N Bolts songs, I'll see if I can import those into BK as well.

My suggestions:

Showdown Town
Logbox 720

[Tee-Hee]

Ice, I've made a BK soundfont that uses exported/decompressed wav files along with the game's actual loop points and detune data etc. Here it is, however it lacks a percussion instrument set, and the 'Effects' are all mapped to random keys. I intended to make a soundfont encompassing all Rare's games, hence the name of the soundfont. However I have thus far only been able to rip the samples from BK. (http://www.sendspace.com/file/mczrju)

[mojobojo]

very good job. I cant wait to see it!

[runehero123]

Ice, I've made a BK soundfont that uses exported/decompressed wav files along with the game's actual loop points and detune data etc. Here it is, however it lacks a percussion instrument set, and the 'Effects' are all mapped to random keys. I intended to make a soundfont encompassing all Rare's games, hence the name of the soundfont. However I have thus far only been able to rip the samples from BK.

Nice work! and yeah, I've noticed the lack of the percussion instrument set in BK because any track set to channel 10 (percussion) wouldn't play in BK.

Perhaps, when I get the ROM Expander working I'll let users add their own instrument sets into BK.

[Gold Jinjo]

Nice!

Would the reverse be possible? Extracting some sort of music (MIDI) data from the games? A MIDI exporter of sorts.

[runehero123]

The first version is out now! Here's a link to the download page:

http://archaicstudios.net/forums/showthread.php?p=3921#post3921

Also, I need someone to help match the instrument list in the midi converter to the instruments in that BK Sound Font. I'd do it myself, but I'm too busy working on another BK Rom Tool at the moment. Thanks in advance!


Nice!

Would the reverse be possible? Extracting some sort of music (MIDI) data from the games? A MIDI exporter of sorts.

Probably, depends on how much time I have to work on it.

[Ice Mario]

Ice, I've made a BK soundfont that uses exported/decompressed wav files

I didn't actually want one, I made one years ago. I just thought it would make the importer easier to use. Perhaps you should speak to Rune about working something out. Good luck.

[runehero123]

I didn't actually want one, I made one years ago. I just thought it would make the importer easier to use. Perhaps you should speak to Rune about working something out. Good luck.

I was thinking. Is there a sound plugin(for any n64 emulator) that contains source for the N64 general Midi Sequencer? It would be cool to include a "test your music" button on the importer, so you can listen to what it actually sounds like in the game, before importing(I tried the sound-font and it's not exactly the same.). Of course, I'd wait to implement that until I finish the ROM Expander, and have it locate the soundbanks automatically in the ROM. I downloaded the Ultra HLE source code, so I'll see if that has what I need.

BTW, I've been trying to get your source code for the GE Decompressor to compile in MVC++ 2008 but it's having trouble with the MFC include files. I'm pretty sure the full version of 2008 comes with MFC. What version of VC++ was it compiled with?

[Mew 101]

Dose anyone know of a way I can convert MP3s to MIDIs? I have some songs I want to use, but there all MP3s.

[empires228]

This looks very promising and usefull for future hacks. But Im doing something wrong off that tutorial because the rom wont load after im finished... I think Im putting it in the rom wrong but the tutorial isn't very specific for people who are not familliar with using the Decompressor. If anyone could help id appariciate it.

[Ice Mario]

BTW, I've been trying to get your source code for the GE Decompressor to compile in MVC++ 2008 but it's having trouble with the MFC include files. I'm pretty sure the full version of 2008 comes with MFC. What version of VC++ was it compiled with?

Sub did the GE Decompressor.

I use Zlib for all my stuff.

[Gold Jinjo]

Dose anyone know of a way I can convert MP3s to MIDIs? I have some songs I want to use, but there all MP3s.


There are tools that do it, but it's practically impossible as MP3 and other audio formats are completely different from MIDI.

Most audio formats contain wave data. MIDI isn't an audio format in this respect. That's why MIDI files are small in size, they're no more than sheet music for the computer to interpret.


Think of it like this: your fingers press the keys on a piano, but the sound doesn't actually come from your fingers.

[Ice Mario]

Good explanation Goldy.

This sort of program would work best on an mp3 that has only one instrument and I imagine it would need to have some kind of percussion or discernable beat difference for the convertor to establish a bpm.

[runehero123]

I have a pretty awesome theory here on importing songs as WAV files. Let me know if this would work:

1) From what I've read about WAV files, and how they work on the N64. The data is streamed from ROM to RAM due to the size of the WAV files. I'm guessing this means you could have an instrument WAV of any given size then without it using much memory in RAM?

2) If 1 is true, then we could import a song in the format of a WAV as an instrument. Next, create a midi with a single track that plays the note with imported instrument data for the duration of the WAV.

If it's possible to do that, then imagine the quality music that would be imported. Well, you'd probably end up with a massive ROM size, but whatever.


This looks very promising and usefull for future hacks. But Im doing something wrong off that tutorial because the rom wont load after im finished... I think Im putting it in the rom wrong but the tutorial isn't very specific for people who are not familliar with using the Decompressor. If anyone could help id appariciate it.

Where are you injecting the file(what address), and what's the size of the converted midi file you're importing? Smaller areas like Mumbo's hut tend to crash the ROM. If the song file is too big and writes over other files in the ROM, then it will crash when it tries to access the data that was overwritten by the file you imported. It's usually fine when importing to Spiral Mountain.

[EDIT]
As for midi files, this site has many Video Game songs you can download:

http://vgmusic.com/

[Tom Servo]

Im having trouble going to the address for the video, but well done man. This is HUGE.

[empires228]

I may be using the wrong address. I tied to insert it at
[107C8] wich was listed as 10 Spiral Mountain was I suspos to insert it as the first part? and not that 107C8?
*1526 - D33AF8* - [107C8] - 10 Spiral Mountain

Ok I have to files
The file one is 1.28 KB (1,312 bytes) Its Crystal Caves from DK64 and is listed as a binn file
The other one is 15.0 KB (15,362 bytes) and its the same song and is listed as a file

wich one should I be insearting?

[SubDrag]

BTW, I've been trying to get your source code for the GE Decompressor to compile in MVC++ 2008 but it's having trouble with the MFC include files. I'm pretty sure the full version of 2008 comes with MFC. What version of VC++ was it compiled with?


I think on 03. But it should certainly compile in 08. GE Editor was originally on 03, and now on 08, and that's far more complicated. What error are you getting? Sometimes there's a stdafx issue, need to put it in include on top.

[runehero123]

I may be using the wrong address. I tied to insert it at
[107C8] wich was listed as 10 Spiral Mountain was I suspos to insert it as the first part? and not that 107C8?
*1526 - D33AF8* - [107C8] - 10 Spiral Mountain

Ok I have to files
The file one is 1.28 KB (1,312 bytes) Its Crystal Caves from DK64 and is listed as a binn file
The other one is 15.0 KB (15,362 bytes) and its the same song and is listed as a file

To answer your first question, you're suppose to insert it at the second address(D33AF8). The address you're inserting it at [107C8] is referring to the location of a pointer that points to that file's address. You want to inject your song at D33AF8 and it should work.

Also, you should inject the file that was output by the converter. In other words, use the larger file. The reason that file is larger is that I have yet to write up the compression code, as it's kinda complicated and involves compressing repeating patterns into "loop data" and matching up patterns and compressing those into pattern commands.


I think on 03. But it should certainly compile in 08. GE Editor was originally on 03, and now on 08, and that's far more complicated. What error are you getting? Sometimes there's a stdafx issue, need to put it in include on top.

Nevermind, I got the GEDecompressor code to compile. I've never used MFC before so I was unaware that I had to create a new MFC Dialog Application and then include your source files(Didn't notice it was just functions.). There were still compile errors related to converting a type of STCTSTR(I think that's the type) to const char*.

[SubDrag]

There's some setting that you need to change back to older method. You don't want unicode, you want whatever the older setting is. That lets a lot of issues pass.

You should be able to compile this as a dll, then using C# use the functions in it via dynamicly loading it.

[Coolboyman]

I'm working on better Text Definitions. Here they are so far.

00 = Nothing
01 = Marimba
02 = Strings

05 = Bongos
06 = Flute 1
07 = Flute 2
08 = Saxophone

0A = Bell

0D = Maracas
0E = Tuba

11 = Trombone

13 = Cymbals

15 = Horn
16 = Flute

1A = Bird Chirp

1D = Oboe
1E = Light Twinkle

2C = Banjo

2E = Wind

34 = "Guh Huh!" Faster

3A = Pick Bass

42 = Underwater Note
43 = "Guh Huh!"

4C = Xylophone

50 = Steel Drum

[runehero123]

Thanks for the updated instruments CBM. I'll update them in the application when I get the time. I'm also working on a possible fix for the hanging note issue.

Also Sub, I'll look into changing the settings back to an older version in MVC++.

[Ice Mario]

As most GM banks all share the same type of instrument layout it might be worth having a "best guess" option that ties each channel to the nearest instrument. Or is that the plan?

[runehero123]

As most GM banks all share the same type of instrument layout it might be worth having a "best guess" option that ties each channel to the nearest instrument. Or is that the plan?

Yeah, I was going to do something like that once we get better definitions of BK's instruments.

[mojobojo]

Very Nice tool. I like it alot!

[runehero123]

UPDATE! v1.02

WHAT's NEW:

~v1.02~
-Fixed bug causing improper placement of note duration values which lead to notes hanging.
-Nullified some controller change commands that weren't supported by Banjo-Kazooie (causing note hang). May not have all.
-Updated instrument definitions (Thanks to Coolboyman).

In a nutshell, there should be less note-hanging in this version. So more midi files should export properly. Unfortunately, Banjo-Kazooie's instruments don't support all the controller status bytes, so it will take time to figure out which ones I'll have to "null" in order to prevent all note hanging errors.

You can download in the original post.

[Coolboyman]

Here's two samples I did of Banjo-Tooie tracks. I had to disable one channel on the Isle O Hags one because it started too early, but I suppose it'll be fixed later.

[runehero123]

Nice, downloaded and will test them out later tonight. Glitter Gulch Mine should be cool.


I had to disable one channel on the Isle O Hags one because it started too early, but I suppose it'll be fixed later.

Was this a problem as a midi file, or after converting? I had a problem with one of the DKC2 midi files I downloaded from vgmusic.com . For some reason, it started out slow (which doesn't occur in the original midi), then sped up as it reached a certain point in the song. At that point, the tracks were all playing at different speeds, so it was pretty weird.

Anyways, I think that had something to do with the midi file and not the converter.

[Breegullbeak]

Does this allow MDI exporting?

[runehero123]

Does this allow MDI exporting?

Not yet, but give me a while and it will be done.

[Breegullbeak]

Not yet, but give me a while and it will be done.

Yeah! Then I can listen to the Treasure Trove Cove them for ever, and ever, an ever, and ever, and...

[SubDrag]

Looks like BK's midis are from:
0D25578
to 0D84440

[mojobojo]

Con someone post a mirror? The link doesn't work. :(

[runehero123]

Yeah, here's the link: http://www.mediafire.com/download.php?ic4c4tngvov

It's also updated on the Youtube demonstration video.

[mojobojo]

Yeah, here's the link: http://www.mediafire.com/download.php?ic4c4tngvov

It's also updated on the Youtube demonstration video.

Awesome, thanks.

[SubDrag]

I decided to go ahead and extract the original midis after finally putting in the wrench time to do it for GE/PD. The instruments are wrong, but somebody with a midi tool can improve.

runehero let me know if you need anything or want me to do DK, etc, just need listing of file spot.

http://www.battlefieldforever.com/hackers/priv/subdrag/bk%20original%20midis.rar

[mojobojo]

I decided to go ahead and extract the original midis after finally putting in the wrench time to do it for GE/PD. The instruments are wrong, but somebody with a midi tool can improve.

runehero let me know if you need anything or want me to do DK, etc, just need listing of file spot.

http://www.battlefieldforever.com/hackers/priv/subdrag/bk%20original%20midis.rar

Thanks, Ive been using with OpenMPT and is working great!

[SubDrag]

Here are JFGs (which are quite fantastic):
http://www.battlefieldforever.com/hackers/priv/subdrag/jfgmidis.rar

[runehero123]

Sub, I got you a list of the DK64 midi file locations. Put together by CBM and myself a while ago.


-Music-
102010 - 0001 - Intro
1035F6 - 0002 - Cranky's Lab
103C0c - 0003 - Minecart race
105012 - 0004 - Boss 1
10633A - 0005 - JJ - Cave
106B64 - 0006 - Funky
1072A6 - 0007 - Ammo Dude
107300 - 0008 - Mini Game
107F7A - 0009 - Chunky - Triangle
10806C - 000A - Diddy - Guitar
10829A - 000B - Ditty
108372 - 000C - Lanky - Trombone
10851E - 000D - Tiny - Saxaphone
10860C - 000E - Angry Aztec
1098D0 - 000F - Transformation
109CBE - 0010 - Tiny Kong - Small
10A16C - 0011 - Chunky - Hunky Chunky
10A530 - 0012 - DONKEY KONG
10A6D8 - 0013 - Angry Aztec - Slide
10B574 - 0014 - Ahhh, Banana
10B668 - 0015 - Angry Aztec - Cave
10BF48 - 0016 - Beta Collection
10C04E - 0017 - Collection - Coin
10C0D4 - 0018 - Collection
10C16E - 0019 - Angry Aztec - Boss
?????? - 001A - Angry Aztec - Inside the Pyramid "GET OUT!"
10D6A6 - 001B - Frantic Factory - Car Race
10DD9C - 001C - Frantic Factory
10E74C - 001D - Snydes Lair
10F030 - 001F - Candy's House
10F598 - 0020 - Collection
10F8B0 - 0021 - Collection
110288 - 0022 - Pause Screen
110316 - 0023 - Collection
110378 - 0024 - Rambi
11069E - 0025 - Angry Aztec - Inside a Pyramid
110700 - 0026 - Water Dropplets
110E6C - 0027 - Frantic Factory - Boss
11129C - 0028 - Achievement
111614 - 0029 - Pushed Start
112E92 - 002A - You screwed up!
112FD4 - 002B - Entrance Down
113026 - 002C - Entrance Up
1131D8 - 002D - Ditty
11332A - 002E - Pleasant Strum
11347C - 002F - Collection
11354E - 0030 - Angry Aztec - Tiny's Fight
113636 - 0031 - Frantic Factory - Some room
11411E - 0032 - Jungle Japes - Barrel Shoot
114920 - 0033 - Frantic Factory - Control Room
115248 - 0034 - Frantic Factory - Crusher Shed
11586E - 0035 - Troff n Scoff
115E52 - 0036 - Donkey's Fanfare
116368 - 0037 - Angry Aztec - Barrel Shoot
116FBA - 0038 - Gloomy Galeon
118BC0 - 0039 - Boss Door Fanfare
118F0A - 003A - Boss Door is now open
11921E - 003B - Sparkles
119336 - 003C - Gloomy Galeon - Boss
11AE7E - 003D - Donkey Kong in style of Saucer Peril
11BA94 - 003E - Gloomy Galeon - Cave
11C516 - 003F - Gloomy Galeon - Inside ship
11C918 - 0040 - Battle
11D6F4 - 0041 - Collection
11D7B0 - 0042 - Fairy
11D97E - 0043 - Collection
11DA24 - 0044 - Fungi Forest - Day
11EE38 - 0045 - Blueprint Collection
11EF1C - 0046 - Fungi Forest - Night
11FCA0 - 0047 - Donkey - Invincible
1203B2 - 0048 - Diddy - Barell Blast
120E04 - 0049 - Lanky - Fast
121A0E - 004A - Fungi Forest - Mine Cart
123098 - 004B - DK Rap
1242F6 - 004C - Found Blueprint
124404 - 004D - Fungi Forest - Inside Barn
124806 - 004E - Gloomy Galeon - ?
124AC4 - 004F - Gloomy Galeon - Inside chest
124DCC - 0050 - Gloomy Galeon - Mermaid
1252B0 - 0051 - Fungi Forest - Boss
12664C - 0052 - Mine Game - Maze
127222 - 0053 - Crystal Caves
127C0A - 0054 - Crystal Caves - Icicles coming down
127DA8 - 0055 - Jungle Noises
127F1E - 0056 - Beat Mini Game
12826A - 0057 - Lose Mini GAme
128568 - 0058 - Mini Game
1286F2 - 0059 - Crystal Caves - Inside Cabin
128B5A - 005A - Mini Game - Minecart Mayahem
12989E - 005B - Gloomy Galeon - Shoot the hearts
129F60 - 005C - Gloomy Galeon - Barell Shoot
12AB62 - 005D - ?
12B350 - 005E - Crystal Caves - ?
12B6C8 - 005F - Crystal Caves - ?
12B9AE - 0060 - Seagulls
12C360 - 0061 - Fungi Forest - Spide
12CC68 - 0062 - Fungi Forest - Cave
12D064 - 0063 - Fungi Forest - ?
12D346 - 0064 - K Rool Battle Warmup
12D7AC - 0065 - Choosing Kong
12DF46 - 0066 - Crystal Caves - Race
12E9BE - 0067 - Crystal Caves - Igloo
12EDEE - 0068 - ?
13F7D8 - 0069 - Creepy Castle
130CC4 - 006A - ?
1320C2 - 006B - Cant Remember This One
1328CE - 006C - Chunky - Invisible
132A2E - 006D - DK Isles
1331AA - 006E - DK Isles - Krem Ship
133852 - 006F - DK Isles - Banana fairy
133CA6 - 0070 - DK Isles - K Lumsy Cave
1342DE - 0071 - Hideout Helm
134DAA - 0072 - Learning New Move
134FAC - 0073 - ?
135080 - 0074 - ?
135B88 - 0075 - Hideout Helm - Minigame
136916 - 0076 - Crystal Caves - Cabin
136CAC - 0077 - Crystal Caves - Cabin
1370BA - 0078 - Crystal Caves - Game
1376D2 - 0079 - Creepy Castle - Inside
137C24 - 007A - First Cutscene
13AA02 - 007B - Donkey Kong's Home
13AEB6 - 007C - Enguarde
13B7B0 - 007D - K Lumsy is a happy chap
13B9B6 - 007E - Creepy Castle Room
13BF58 - 007F - Music Refill
13C0E4 - 0080 - Collection
13C168 - 0081 - Creepy Castle - In somewhere?
13C6CC - 0082 - DK Isles - Angry Aztec
13CAF0 - 0083 - DK Isles - Jungle Japes
13D11A - 0084 - DK Isles - Frantic Factory
13D63C - 0085 - DK Isles - Gloomy Galeon
13DA0A - 0086 - Donkey Kong!
13E334 - 0087 - Creepy Castle - In somewhere?
13E76A - 0088 - Creepy CAstle - Museum
13EF62 - 0089 - Creepy Castle - ?
13F3E4 - 3008A - Game Over
13F60C - 008B - Creepy Castle - ?
13FABC - 008C - Creepy Castle - Tower Room
140B8E - 008D - Creepy Castle - ?
140EC4 - 008E - Creepy Castle - ?
1412B6 - 008F - Collection
141510 - 0090 - Collection
14157C - 0091 - Collection
141678 - 0092 - Collection
14187C - 0093 - Collection
141A8A - 0094 - Crystal Caves - Boss
1437FC - 0095 - Creepy Castle - Boss
144D42 - 0096 - Creepy Castle - ?
1451EE - 0097 - Ditty
14547E - 0098 - K Rool Fight
147698 - 0099 - Donkey Kongs Advent
147A80 - 009A - DK Isles - Crystal Caves
147F74 - 009B - DK Isles - Creepy Castle
1481F4 - 009C - DK Isles - Hideout Helm
148584 - 009D - ?
148CBE - 009E - Very Final
149DD0 - 009F - Life's a Beach
14ADD4 - 00A0 - Jungle Japes
14BBC0 - 00A1 - Jungle Japes - Rain
14C202 - 00A2 - DK Isles - ?
14D342 - 00A3 - Crystal Caves - Barell
14DD18 - 00A4 - Fungi Forest - Barell
14E64C - 00A5 - Creepy Castle - Barell
14F06A - 00A6 - DK Isles - Snyde
14F474 - 00A7 - Final Fight Fanfare
14FB72 - 00A8 - ?
1506CC - 00A9 - Fungus Forest - Race
15122A - 00AA - Game Over
151968 - 00AB - Wrinkley
151C22 - 00AC - Got all Bananas on one world
151D36 - 00AD - Up Down Ditty
1520AC - 00AE - Copyright Info

[SubDrag]

10B668, "15Angry AztecCave is incorrect
13F7D8, "69Creepy Castle is incorrect

What about Kiosk?
http://www.battlefieldforever.com/hackers/priv/subdrag/dkmidi.rar

DKs sound great!

[SubDrag]

BT locations?

[runehero123]

According to Coolboyman, the midi files start at 0x1339134.

[SubDrag]

Formats a little different on these: It's division, then # tracks, then tracks, then data. OK I'll grab 'em

[SubDrag]

Here's BT:
http://www.battlefieldforever.com/hackers/priv/subdrag/btmidis.rar

[SubDrag]

Here's Conker:
was uncompressed in ROM oddly
http://www.battlefieldforever.com/hackers/priv/subdrag/conker%20original%20midis.zip

In such a huge ROM...

[runehero123]

Here's Conker:
was uncompressed in ROM oddly
http://www.battlefieldforever.com/ha...al%20midis.zip

In such a huge ROM...

Yeah, the midi files in DK64 kiosk are uncompressed as well. They were compressed in the final version of course to save size. I'm guessing they had enough room to spare on the cart since the songs weren't compressed in conker.

Anyways, here's where the Kiosk version starts: 15ACED0

The strange thing here is that all the songs that weren't used are replaced with the same track, instead of blank data.

[Mr Data]

Quote by Tee Hee:

Ice, I've made a BK soundfont that uses exported/decompressed wav files along with the game's actual loop points and detune data etc. Here it is, however it lacks a percussion instrument set, and the 'Effects' are all mapped to random keys. I intended to make a soundfont encompassing all Rare's games, hence the name of the soundfont. However I have thus far only been able to rip the samples from BK.



So how is the bk soundfont going? I hope it isn't discontinued/canceled so fans can make banjo kazooie/tooie remixes galore and better midis. Also you'd be able to make original midis for fan games/projects.