|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
;certutil -config "MYCA.info.sys\IssuingCAInstance" -ping ;certreq -new certreq.inf certreq.req ;certreq -submit -config "MYCA.info.sys\IssuingCAInstance" certreq.req ;certreq -accept c:\MyGeneratedCert.cer ;netsh http add sslcert ipport=0.0.0.0:5000 certhash=THUMBPRINT appid={} certstorename=my ;netsh http add sslcert ipport=0.0.0.0:5000 certhash=d9ffaeec6552b3d21d3f7741c48d63d9fb61d10a appid={01122334-4556-6778-8990-aabbccddeeff} certstorename=my [NewRequest] ; The subject name for the certificate (Common Name) Subject = "CN=MYCN.info.sys" KeySpec = 1 KeyLength = 2048 Exportable = TRUE MachineKeySet = TRUE ; Use the appropriate Cryptographic Service Provider for your environment ProviderName = "Microsoft RSA SChannel Cryptographic Provider" RequestType = PKCS10 [Extensions] ; The Enhanced Key Usage extension for Server Authentication 2.5.29.37 = "{text}1.3.6.1.5.5.7.3.1" ; The Subject Alternative Name (SAN) extension 2.5.29.17 = "{text}" ; List multiple DNS entries (ampersand-& delimited) _continue_ = "DNS=DNSName&" _continue_ = "DNS=DNSNAME.contoso.sys&" _continue_ = "DNS=DNSNAME2&" _continue_ = "DNS=DNSNAME2.contoso.sys" [RequestAttributes] ; Specify the certificate template name here CertificateTemplate = "WebServer_Contoso" |
C# Discord bot for .NET 8.0
In order to get this to run on a non-dev runtime box, you’ll need to install multiple of not all of these runtimes located here: https://learn.microsoft.com/en-gb/cpp/windows/latest-supported-vc-redist?view=msvc-170
I think the 2005VCRuntime along with the 2015-2019VCRuntime got it working on my Server 2019 Core setup.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 |
// See https://aka.ms/new-console-template for more information //leaderboards //https://huggingface.co/spaces/mteb/leaderboard //Implement with Python //https://www.llamaindex.ai/ //https://docs.llamaindex.ai/en/stable/getting_started/discover_llamaindex/ //Install-Package LLamaSharp //Install-Package LLamaSharp.Backend.Cuda11 //https://dotnet.microsoft.com/en-us/download/dotnet/8.0 //https://dotnet.microsoft.com/en-us/download/dotnet //The top line returns waaayy better results then the bottom //Assistant: discovery\r\nUser: //discovery using LLama.Common; using LLama; #pragma warning disable SKEXP0050, SKEXP0001, SKEXP0020 //RAG //Install-Package LLamaSharp.Semantic-Kernel //https://github.com/SciSharp/LLamaSharp/issues/648 using LLamaSharp.SemanticKernel.TextEmbedding; //Install-Package Microsoft.SemanticKernel using Microsoft.SemanticKernel.Connectors; using Microsoft.SemanticKernel.Memory; using Microsoft.SemanticKernel.Text; //Install-Package Microsoft.SemanticKernel.Connectors.Sqlite -Version 1.18.0-alpha using Microsoft.SemanticKernel.Connectors.Sqlite; // using System.IO; using System.Text.RegularExpressions; using System.Diagnostics; using System.Net.WebSockets; using System.Text; using System.Net; using System.Management; using SQLitePCL; using System; using System.Collections; using System.Windows.Documents; using System.Runtime.InteropServices; using static System.Net.Mime.MediaTypeNames; using System.CodeDom.Compiler; using System.Reflection; using Microsoft.Data.Sqlite; using static System.Collections.Specialized.BitVector32; using LLama.Native; //LLava //Install-package spectre.console [DllImport("user32.dll")] static extern IntPtr SetForegroundWindow(IntPtr hWnd); [DllImport("user32.dll")] static extern bool ShowWindow(IntPtr hWnd, int nCmdShow); //ShowWindow needs an IntPtr Console.ForegroundColor = ConsoleColor.White; Console.WriteLine("Starting ISPortal AI!"); HttpListenerWebSocketContext webSocketContext = null; //https://huggingface.co/mradermacher/llava-v1.6-mistral-7b-GGUF/tree/main string RagModelPath = @"D:\LLM\bge-large-en-v1.5-q8_0.gguf"; //string modelPath = @"D:\Meta-Llama-3.1-8B-Instruct-Q6_K.gguf"; string modelPath = @"D:\LLM\Saul-Instruct-v1.Q8_0.gguf"; //string modelPath = @"D:\Meta-Llama-3.1-8B-Instruct-Q8_0.gguf"; // change it to your own model path. //string modelPath = @"D:\Meta-Llama-3.1-70B-Instruct-Q6_K-00001-of-00002.gguf"; // Way too large for my computer //var embeddingModelPath = Path.GetFullPath("<path to embed model>/nomic-embed.gguf"); string LocalMemoryPath = @"C:\Users\User\Desktop\MyMemory.db"; string PathToFilesToIngest = "D:\\Embeddings"; ManagementObjectSearcher searcher = new ManagementObjectSearcher("SELECT * FROM Win32_Processor"); List<int> coresCPUs = new List<int>(); foreach (ManagementObject mo in searcher.Get()) { coresCPUs.Add(Convert.ToInt32(mo.Properties["NumberOfCores"].Value)); Debug.WriteLine(Convert.ToInt32(mo.Properties["NumberOfCores"].Value)); } //RAG variables ModelParams RAGparameters = null; ISemanticTextMemory MyMemoryStore = null; //All operations LLamaWeights model = null; LLamaContext context = null; InteractiveExecutor executor = null; var MyStartupMethod = MyLoadedMethod.RAG; string collectionName = "TestCollection"; // Add chat histories as prompt to tell AI how to act. var chatHistory = new ChatHistory(); var chatHistory2 = new ChatHistory(); string Str = null; switch ("rag") { case "train": chatHistory.AddMessage(AuthorRole.System, "Generate new examples based on user input examples, once asked to write additional samples, take the previous request and continue to generate addtional samples. All answers need to strictly be formed into a statement and then answered as a JSON array of strings, always add a semicolon after each string array to close the statement properly. All commands should be for use by the windows operating system. Do not explain, add notes or say anything else unless specifically asked to for any reason."); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate to program files in windows, find a folder with the word steam in it then create a subfolder inside with two text files and populate a random string in each text file. Write the commands using a json array of strings to be processed by an interrupter."); //10000 Str = "myArray = [\"CD \\\\\", \"CD C:\\\\Program Files\\\\\", \"DIR /OD \\\"*steam*\\\"\", \"MKDIR \\\"SteamFolder\\\"\", \"MD \\\"SteamFolder\\\\subfolder\\\", \\\"ECHO New content 1 > \\\"SteamFolder\\\\subfolder\\\\file1.txt\", \"COPY \\\"asdawdafag\\\" > \\\"SteamFolder\\\\subfolder\\\\file2.txt\\\"\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate list the files and folders in the C: drive, the shutdown the computer in 60 seconds."); //10000 Str = "myArray = [\"DIR C:\\\", \"shutdown -s -t 60\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "Great, now continue to write addtional samples, remember to keep the format as a statement, then followed by a JSON answer with a semicolon at the end of each JSON answer. I will ask for over 100 samples."); //10000 chatHistory.AddMessage(AuthorRole.Assistant, "Ping google.com and write the the results to a text file then open the program files folder using explorer. myArray = [\"ping google > c:\\Output.txt\", \"Explorer \"%programfiles%\"\"];"); //10000 chatHistory.AddMessage(AuthorRole.User, "y"); //10000 chatHistory.AddMessage(AuthorRole.Assistant, "write a list of commands that searches for files in the D: Drive that start with the name \"test\" and then copy those files to the c: drive if they contain a word \"echo\" inside. myArray = [\"DIR /S /B /A:-D D:\\test*.txt\", \"COPY \\\"D:\\\\test*.txt\\\" C:\\\" /Y\", \"FINDSTR /C:\"\"echo\"\" D:\\\\test*.txt > E:\\\\log.txt\"];"); //10000 chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Use Moveit.exe to move a file named NTFS.mbr localy, then zip the moved file and display its size. myArray = [\"Moveit \\\"NTFS.mbr\\\" C:\", \"7z a ntfs.zip ntfs.mbr\", \"7z l ntfs.zip\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Write a list of commands to direct a LAN IP to a temp directory, download a single file using curl and extract the extracted content into a subfolder named debug. myArray = [\"MKDIR C:\\\\tmp\\\", \"curl \\\"http://192.168.1.100/somefile.zip\\\" -o C:\\\\tmp\", \"7z e C:\\\\tmp\\\\somefile.zip -od C:\\\\tmp\\\\debug\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Create a random 32 character hexadecimal color code, then use the color picker tool in Windows to open up a default browser using this new color as its background. myArray = [\"wmic os get version > C:\\versions.txt\", \"color cc233330cc2233c2\", \"explorer https://www.google.com\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Write a list of commands to create an Ecrypted zip archive and store it in the users desktop directory, than extract this zip file with AES encryption using 7z. myArray = [\"cipher /E /S:C:\\\\Users\\\\username\\\\Documents\" > C:\\UserEnv, \"cipher /D /S:C:\\\\Users\\\\username\\\\Documents\", \"CIPHER /P\"aVeryStrongPassword123\\* \\\"C:\\\\Users\\\\username\\\\Desktop\\\\enc.zip\\\"\", \"7z e -hp\"aVeryStrongPassword123 \\*.zip\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Display user accounts with administrative privileges, and list the user object properties in Windows registry. myArray = [\"wmic useraccount where Enabled=TRUE and Administrator=TRUE get *\", \"reg query HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\\\Winlogon /v DefaultUserName\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Change password for a service called misconfigure-MDViewer, change priority of CpuGroup1; write a list of commands to accomplish these tasks. myArray = [\"net user \\\"misconfigure-MDViewer\\\" changesPassword!!\"], \"wmic cpu Get Name\", \"wmic process where ProcessID=1234 set Priority=5\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Use the Windows Task Scheduler to create a scheduled task to download a simple text file from a remote server every 10 minutes. myArray = [\"schtasks /create /tn \"MyTask\" /tr \"curl http://192.168.1.100/textfile.txt -o C:\\tmp\\mytext.txt\\\"\", \"schtasks /change /tn \"MyTask\\\" /st 00:00 /ri 10\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Write a batch file that opens up Internet Explorer with two tabs one with the url google.com and another with bing.com. myArray = [\"start iexplore.exe \\\"http://www.google.com\\\", start iexplore.exe \\\"http://www.bing.com\\\"\"];"); break; case "production": chatHistory.AddMessage(AuthorRole.System, "Transcribe request from the user into an array of JSON strings labeled 'myArray' to be processed by an interpreter that will parse an array of strings and execute them on a system. All answers need to strictly be formed into a JSON array of strings for the windows operating system. Only give the results in JSON format and do not explain or say anything else unless asked specifically."); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate to program files in windows, find a folder with the word steam in it then create a subfolder inside with two text files and populate a random string in each text file. Write the commands using a json array of strings to be processed by an interrupter."); //10000 Str = "myArray = [\"CD \\\\\", \"CD C:\\\\Program Files\\\\\", \"DIR /OD \\\"*steam*\\\"\", \"MKDIR \\\"SteamFolder\\\"\", \"MD \\\"SteamFolder\\\\subfolder\\\", \\\"ECHO New content 1 > \\\"SteamFolder\\\\subfolder\\\\file1.txt\", \"COPY \\\"asdawdafag\\\" > \\\"SteamFolder\\\\subfolder\\\\file2.txt\\\"\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate list the files and folders in the C: drive, the shutdown the computer in 60 seconds."); //10000 Str = "myArray = [\"DIR C:\\\", \"shutdown -s -t 60\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "Ping google.com and write the the results to a text file then open the program files folder using explorer."); //10000 Str = "myArray = [\"ping google > c:\\Output.txt\", \"Explorer \"%programfiles%\"\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that searches for files in the D: Drive that start with the name \"test\" and then copy those files to the c: drive if they contain a word \"echo\" inside."); //10000 Str = "myArray = [\"DIR /S /B /A:-D D:\\test*.txt\", \"COPY \\\"D:\\\\test*.txt\\\" C:\\\" /Y\", \"FINDSTR /C:\"\"echo\"\" D:\\\\test*.txt > E:\\\\log.txt\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); break; case "rag": //chatHistory.AddMessage(AuthorRole.System, "You are a legal assistance. You will be provided context to some data and will answer any questions you receive the best you can with the context provided and do not include information outside of the information provided."); break; default: Console.WriteLine("Number is not between 1 and 4"); break; } //LLM variables var LLMparameters = new ModelParams(modelPath) { ContextSize = 13107, // Does not require alot of memory //ContextSize = 131072, // The longest length of chat as memory. This can bring the computer to a crawl //If commented out, swaps to CPU //GpuLayerCount = 0 GpuLayerCount = 33 // How many layers to offload to GPU. Please adjust it according to your GPU memory. }; LLamaWeights LLMmodel = null; LLamaContext LLMcontext = null; LLamaContext.State CleanContext = null; InteractiveExecutor LLMexecutor = null; StatelessExecutor LLMStatelessExecutor = null; //Not currently in use. Results were bad. StatefulExecutorBase.ExecutorBaseState CleanExectuor = null; ChatSession LLMChatSession = null; SessionState CleanChatSession = null; InferenceParams inferenceParams = new InferenceParams() { Temperature = 0.4f, MaxTokens = 512, // No more than 256 tokens should appear in answer. Remove it if antiprompt is enough for control. AntiPrompts = new List<string> { "\nUser:" } // Stop generation once antiprompts appear. }; async Task LoadModule() { if (MyStartupMethod == MyLoadedMethod.RAG) //RAG //Checks for enum need to be wrapped in a function to be valid. { RAGparameters = new ModelParams(RagModelPath) { ContextSize = 32000, // Does not require alot of memory //ContextSize = 131072, // The longest length of chat as memory. This can bring the computer to a crawl Embeddings = true, //GpuLayerCount = 0 //If commented out, swaps to CPU GpuLayerCount = 33 // How many layers to offload to GPU. Please adjust it according to your GPU memory. }; model = LLamaWeights.LoadFromFile(RAGparameters); context = model.CreateContext(RAGparameters); executor = new InteractiveExecutor(context); var embedder = new LLamaEmbedder(model, RAGparameters); var service = new LLamaSharpEmbeddingGeneration(embedder); MyMemoryStore = new MemoryBuilder().WithMemoryStore(await SqliteMemoryStore.ConnectAsync(LocalMemoryPath)).WithTextEmbeddingGeneration(service).Build(); await IngestFile(); LLMmodel = LLamaWeights.LoadFromFile(LLMparameters); LLMStatelessExecutor = new StatelessExecutor(LLMmodel, LLMparameters); LLMcontext = LLMmodel.CreateContext(LLMparameters); LLMexecutor = new InteractiveExecutor(LLMcontext); LLMChatSession = new(LLMexecutor, chatHistory); } else { LLMmodel = LLamaWeights.LoadFromFile(LLMparameters); LLMcontext = LLMmodel.CreateContext(LLMparameters); LLMexecutor = new InteractiveExecutor(LLMcontext); LLMChatSession = new(LLMexecutor, chatHistory); } CleanExectuor = LLMexecutor.GetStateData(); CleanChatSession = LLMChatSession.GetSessionState(); CleanContext = LLMcontext.GetState(); //_memoryStore //var field = MyMemoryStore.GetType().GetField("_storage", BindingFlags.NonPublic | BindingFlags.Instance); //var memoryStore = (Dictionary<string, string>)field.GetValue(MyMemoryStore); //var keys = memoryStore.Keys.ToList(); using (var connection = new SqliteConnection("Data Source=" + LocalMemoryPath)) { connection.Open(); var command = connection.CreateCommand(); command.CommandText = @"SELECT COUNT(1) from SKMemoryTable"; Debug.WriteLine(command.ExecuteScalar()); //collection, key, metadata, embedding, timestamp //metadata holds a json string of "is_reference", "external_source_name", "id" and "description", "text", "additional_metadata" command.CommandText = @"SELECT DISTINCT key FROM SKMemoryTable"; // Replace 'MemoryTable' with the actual table name using (var reader = command.ExecuteReader()) { while (reader.Read()) { //Console.WriteLine(reader.GetString(0)); } } } //System.Diagnostics.Debugger.Break(); //LLMsession.SaveSession(); } if (false) { await RunLLava(); Console.ReadKey(); return; } await LoadModule(); string CompleteAnswer = null; Match Matches; long TokensInMemory = 0; IEnumerable<string> SplitToLines(string stringToSplit, int maximumLineLength) { var words = stringToSplit.Split(' ').Concat(new[] { "" }); return words .Skip(1) .Aggregate( words.Take(1).ToList(), (a, w) => { var last = a.Last(); while (last.Length > maximumLineLength) { a[a.Count() - 1] = last.Substring(0, maximumLineLength); last = last.Substring(maximumLineLength); a.Add(last); } var test = last + " " + w; if (test.Length > maximumLineLength) { a.Add(w); } else { a[a.Count() - 1] = test; } return a; }); } async Task IngestFile() { long TotalParagraphsLoaded = 0; Console.WriteLine("===== INGESTING ====="); IList<string> collections = await MyMemoryStore.GetCollectionsAsync(); string folderPath = Path.GetFullPath(PathToFilesToIngest); string[] files = Directory.GetFiles(folderPath); if (collections.Contains(collectionName)) { Console.WriteLine("Found database"); } else { List<Task> tasks = new List<Task>(); foreach (var item in files.Select((path, index) => new { path, index })) { Console.WriteLine($"Ingesting file #{item.index}"); string text = File.ReadAllText(item.path); if (false) { var sentences = TextChunker.SplitPlainTextLines(text, 256); var paragraphs = TextChunker.SplitPlainTextParagraphs(sentences, 1024); foreach (var para in paragraphs.Select((text, index) => new { text, index })) await MyMemoryStore.SaveInformationAsync(collectionName, para.text, $"Document {item.path}, Paragraph {para.index}"); } else if (true) { string[] splitQuestions = Regex.Split(text, @"(?=Question\s\d+\.)"); long ParaIndex = 1; foreach (string question in splitQuestions) { if (!string.IsNullOrWhiteSpace(question)) { try { //description is stored in the metadata column, same with additional_metadata. Remember, converting to weights takes about 6-7x times the space in total. Debug.WriteLine("Importing: " + model.Tokenize(question.Trim(), true, false, Encoding.ASCII).Length + "\r\n"); if (model.Tokenize(question.Trim(), true, false, Encoding.ASCII).Length >= 512) { foreach (string DataToAdd in SplitToLines(question.Trim(), question.Trim().Length / 2 + 10)) //800 { Debug.WriteLine("Splitting into: " + model.Tokenize(DataToAdd, true, false, Encoding.ASCII).Length + "\r\n"); await MyMemoryStore.SaveInformationAsync(collectionName, DataToAdd, ($"Document {item.path}, Paragraph {ParaIndex++}")); } } else { await MyMemoryStore.SaveInformationAsync(collectionName, question.Trim(), ($"Document {item.path}, Paragraph {ParaIndex++}")); } } catch { } //tasks.Add(Task.Run(() => memory.SaveInformationAsync(collectionName, question.Trim(), $"Document {item.path}, Paragraph {ParaIndex++}"))); TotalParagraphsLoaded++; } } } else if (false) { string[] splitQuestions = Regex.Split(text, @"(?<=[.!?])\s+|(?<=\.\.)\s*|\r\n|\n"); //each sentence long ParaIndex = 1; foreach (string question in splitQuestions) { if (!string.IsNullOrWhiteSpace(question)) { //await memory.SaveReferenceAsync() await MyMemoryStore.SaveInformationAsync(collectionName, question.Trim(), $"Document {item.path}, Paragraph {ParaIndex++}"); //tasks.Add(Task.Run(() => memory.SaveInformationAsync(collectionName, question.Trim(), $"Document {item.path}, Paragraph {ParaIndex++}"))); TotalParagraphsLoaded++; } } } //await Task.WhenAll(tasks); } Console.WriteLine("Generated database with " + TotalParagraphsLoaded + " items."); } Console.WriteLine("===== DONE INGESTING ====="); } void ClearContext() { return; LLMChatSession.LoadSession(CleanChatSession); Debug.WriteLine(LLMChatSession.HistoryTransform.HistoryToText(LLMChatSession.History)); return; LLMcontext.Dispose(); LLMexecutor = null; LLMChatSession = null; LLMcontext = LLMmodel.CreateContext(LLMparameters); LLMexecutor = new InteractiveExecutor(LLMcontext); LLMChatSession = new(LLMexecutor, chatHistory); return; chatHistory.Messages.Clear(); LLMcontext.LoadState(CleanContext); LLMChatSession.LoadSession(CleanChatSession); LLMexecutor.LoadState(CleanExectuor); return; while (LLMChatSession.History.Messages.Count > 0) { LLMChatSession.RemoveLastMessage(); } return; } string Keywords(string sentence) { HashSet<string> StopWords = new HashSet<string>(new[] { "a", "an", "and", "the", "is", "in", "on", "at", "of", "for", "to", "with", "by", "from", "that", "this", "it", "as", "are", "was", "were", "be", "been", "has", "have", "had", "but", "or", "not", "so", "if", "then", "i", "me", "my", "myself", "we", "our", "ours", "ourselves", "you", "your", "yours", "yourself", "yourselves", "he", "him", "his", "himself", "she", "her", "hers", "herself", "its", "itself", "they", "them", "their", "theirs", "themselves", "what", "which", "who", "whom", "these", "those", "am", "being", "do", "does", "did", "doing", "because", "until", "while", "about", "against", "between", "into", "through", "during", "before", "after", "above", "below", "up", "down", "out", "off", "over", "under", "again", "further", "once", "here", "there", "when", "where", "why", "how", "all", "any", "both", "each", "few", "more", "most", "other", "some", "such", "no", "nor", "only", "own", "same", "than", "too", "very", "s", "t", "can", "will", "just", "don", "should", "now" }); string cleanSentence = Regex.Replace(sentence.ToLower(), @"[^\w\s]", ""); // Split sentence into words var words = cleanSentence.Split(new[] { ' ', '\t', '\n', '\r' }, StringSplitOptions.RemoveEmptyEntries); // Filter out stop words and return remaining words as keywords var keywords = words.Where(word => !StopWords.Contains(word)).ToList(); // Join the keywords into a single string separated by spaces return string.Join(" ", keywords); } async void ProcessAIRequest(string userInput) { Debug.WriteLine("ProcessAIRequest starting"); TokensInMemory += model.Tokenize(userInput, true, false, Encoding.ASCII).Length; //executor.rag //RAG and context building: https://medium.com/@bruno.facco/rag-from-concept-to-application-using-gpt-llama-index-and-sql-retriever-cfb437da99eb //session. String ChatResponseToMeasure = null; if (MyStartupMethod == MyLoadedMethod.RAG) { StringBuilder ColoredStringBuilder = new(); ColoredStringBuilder.Clear(); List<string> sources = []; var resultsfound = false; double minRelevanceScoreValue = 0.8; Console.WriteLine("=== Extracting Keywords ==="); /* string prompt = "System: Ensure that the results contain ALL of the words from the following user query. The words must appear at least once anywhere in the result, without exception.\r\n The results are in the following format, The question number, the question, possible answers then followed by the explanation. The user will give keywords and return the results if those keywords appear in any of the result, either in the question or the explanation.\r\n" + "Assistant: I will only return results that include every single keyword provided by the user. If even one keyword is missing, the result will not be returned, without exception.\r\n" + "User: " + userInput; */ string prompt = userInput; string keywords = ""; string StrchatHistory = "\r\nUser: What is Congresses Commerce Clause powers?\r\n" + "Assistant: Congress Commerce Clause\r\n" + "User: What time is it in new york?\r\n" + "Assistant: time new york\r\n" + "User: How do you fill up a car tire?\r\n" + "Assistant: fill up car tire\r\n" + "User: How fast can a car go on a freeway?\r\n" + "Assistant: fast car freeway\r\n" + "User: How do you make pancakes in an oven?\r\n" + "Assistant: make pancakes oven\r\n" + "User: How do you write a business plan?\r\n" + "Assistant: write business plan\r\n" + "User: What is the capital of Japan?\r\n" + "Assistant: capital Japan\r\n" + "User: How to set up a new email account?\r\n" + "Assistant: set up email account\r\n" + "User: What are the benefits of meditation?\r\n" + "Assistant: benefits meditation\r\n" + "User: How to make a website responsive?\r\n" + "Assistant: make website responsive\r\n" + "User: What is the difference between Python and Java?\r\n" + "Assistant: difference Python Java\r\n" + "User: How do I replace a car battery?\r\n" + "Assistant: replace car battery\r\n" + "User: What is the tallest mountain in the world?\r\n" + "Assistant: tallest mountain world\r\n" + "User: How can I improve my public speaking skills?\r\n" + "Assistant: improve public speaking skills\r\n" + "User: What is the population of Canada?\r\n" + "Assistant: population Canada\r\n" + "User: How to brew coffee using a French press?\r\n" + "Assistant: brew coffee French press\r\n" + "User: How does a blockchain work?\r\n" + "Assistant: blockchain work\r\n" + "User: What are the side effects of aspirin?\r\n" + "Assistant: side effects aspirin\r\n" + "User: How to start learning data science?\r\n" + "Assistant: start learning data science\r\n" + "User: What is the speed of light?\r\n" + "Assistant: speed of light\r\n" + "User: How do I bake a chocolate cake?\r\n" + "Assistant: bake chocolate cake\r\n" + "User: What are the causes of climate change?\r\n" + "Assistant: causes climate change\r\n" + "User: How to play the guitar for beginners?\r\n" + "Assistant: play guitar beginners\r\n" + "User: What are the symptoms of a common cold?\r\n" + "Assistant: symptoms common cold\r\n" + "User: How to create a budget plan?\r\n" + "Assistant: create budget plan\r\n"; if (false) { if (true) { await foreach (var text in LLMStatelessExecutor.InferAsync("System: Return only the keywords from the following statement and do not provide anything else. Use spaces when outputing the keywords values and do not add anything futher." + StrchatHistory + "User: '" + userInput + "'", inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); keywords += text; } } else { //chatHistory.Messages.Clear(); chatHistory.AddMessage(AuthorRole.System, "I am a search engine assistant. I will only return back keywords from any statement you make and nothing else with no exceptions.\r\n"); chatHistory.AddMessage(AuthorRole.User, "What is Congresses Commerce Clause powers?"); chatHistory.AddMessage(AuthorRole.Assistant, "Congress Commerce Clause"); chatHistory.AddMessage(AuthorRole.User, "What time is it in new york?"); chatHistory.AddMessage(AuthorRole.Assistant, "time new york"); chatHistory.AddMessage(AuthorRole.User, "How do you fill up a car tire?"); chatHistory.AddMessage(AuthorRole.Assistant, "fill up car tire"); chatHistory.AddMessage(AuthorRole.User, "How fast can a car go on a freeway?"); chatHistory.AddMessage(AuthorRole.Assistant, "fast car freeway"); chatHistory.AddMessage(AuthorRole.User, "How do you make pancakes in an oven?"); chatHistory.AddMessage(AuthorRole.Assistant, "make pancakes oven"); chatHistory.AddMessage(AuthorRole.User, "How do you write a business plan?"); chatHistory.AddMessage(AuthorRole.Assistant, "write business plan"); chatHistory.AddMessage(AuthorRole.User, "What is the capital of Japan?"); chatHistory.AddMessage(AuthorRole.Assistant, "capital Japan"); chatHistory.AddMessage(AuthorRole.User, "How to set up a new email account?"); chatHistory.AddMessage(AuthorRole.Assistant, "set up email account"); chatHistory.AddMessage(AuthorRole.User, "What are the benefits of meditation?"); chatHistory.AddMessage(AuthorRole.Assistant, "benefits meditation"); chatHistory.AddMessage(AuthorRole.User, "How to make a website responsive?"); chatHistory.AddMessage(AuthorRole.Assistant, "make website responsive"); chatHistory.AddMessage(AuthorRole.User, "What is the difference between Python and Java?"); chatHistory.AddMessage(AuthorRole.Assistant, "difference Python Java"); chatHistory.AddMessage(AuthorRole.User, "How do I replace a car battery?"); chatHistory.AddMessage(AuthorRole.Assistant, "replace car battery"); chatHistory.AddMessage(AuthorRole.User, "What is the tallest mountain in the world?"); chatHistory.AddMessage(AuthorRole.Assistant, "tallest mountain world"); chatHistory.AddMessage(AuthorRole.User, "How can I improve my public speaking skills?"); chatHistory.AddMessage(AuthorRole.Assistant, "improve public speaking skills"); chatHistory.AddMessage(AuthorRole.User, "What is the population of Canada?"); chatHistory.AddMessage(AuthorRole.Assistant, "population Canada"); chatHistory.AddMessage(AuthorRole.User, "How to brew coffee using a French press?"); chatHistory.AddMessage(AuthorRole.Assistant, "brew coffee French press"); chatHistory.AddMessage(AuthorRole.User, "How does a blockchain work?"); chatHistory.AddMessage(AuthorRole.Assistant, "blockchain work"); chatHistory.AddMessage(AuthorRole.User, "What are the side effects of aspirin?"); chatHistory.AddMessage(AuthorRole.Assistant, "side effects aspirin"); chatHistory.AddMessage(AuthorRole.User, "How to start learning data science?"); chatHistory.AddMessage(AuthorRole.Assistant, "start learning data science"); chatHistory.AddMessage(AuthorRole.User, "What is the speed of light?"); chatHistory.AddMessage(AuthorRole.Assistant, "speed of light"); chatHistory.AddMessage(AuthorRole.User, "How do I bake a chocolate cake?"); chatHistory.AddMessage(AuthorRole.Assistant, "bake chocolate cake"); chatHistory.AddMessage(AuthorRole.User, "What are the causes of climate change?"); chatHistory.AddMessage(AuthorRole.Assistant, "causes climate change"); chatHistory.AddMessage(AuthorRole.User, "How to play the guitar for beginners?"); chatHistory.AddMessage(AuthorRole.Assistant, "play guitar beginners"); chatHistory.AddMessage(AuthorRole.User, "What are the symptoms of a common cold?"); chatHistory.AddMessage(AuthorRole.Assistant, "symptoms common cold"); chatHistory.AddMessage(AuthorRole.User, "How to create a budget plan?"); chatHistory.AddMessage(AuthorRole.Assistant, "create budget plan"); await foreach (var text in LLMChatSession.ChatAsync(new ChatHistory.Message(AuthorRole.User, userInput), inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); keywords += text; } } } keywords = Keywords(userInput); Console.WriteLine(keywords); while (!resultsfound) { Console.WriteLine("=== RETRIEVING [" + minRelevanceScoreValue + "]==="); //"Instruct:Given the query, find all relevant text.\r\nQuery: " + keywords + "\r\n" //"Assistant: " + keywords + "\r\nUser:" //await foreach (var result in MyMemoryStore.SearchAsync(collectionName, "Query: " + keywords + "\r\n", limit: 10, minRelevanceScore: minRelevanceScoreValue)) await foreach (var result in MyMemoryStore.SearchAsync(collectionName, "Assistant: " + keywords + "\r\nUser:", limit: 10, minRelevanceScore: minRelevanceScoreValue)) { resultsfound = true; //builder.AppendLine(result.Metadata.Text + "\r\n\r\n" + result.Relevance + "\r\n\r\n"); //CompleteAnswer += result.Metadata.Text; string highlightedText = result.Metadata.Text; foreach (var word in keywords.Split(' ', StringSplitOptions.RemoveEmptyEntries)) { highlightedText = Regex.Replace(highlightedText, " " + word, match => $"\x1b[31m{match.Value}\x1b[0m", RegexOptions.IgnoreCase); } CompleteAnswer += result.Metadata.Text + "\r\n"; ColoredStringBuilder.AppendLine(highlightedText + "\r\n\r\n" + result.Relevance + "\r\n\r\n"); sources.Add(result.Metadata.Id); } minRelevanceScoreValue -= .05d; if (minRelevanceScoreValue < .2d) { Console.WriteLine("Unable to find any subject using your keywords"); return; } } ColoredStringBuilder.AppendLine(""" Sources: """); CompleteAnswer += ("\r\n\r\nSources:\r\n"); foreach (string source in sources) { ColoredStringBuilder.AppendLine($" {source}"); CompleteAnswer += $" {source}\r\n"; } CompleteAnswer += "\r\n"; Console.WriteLine("=== DONE RETRIEVING ==="); Console.WriteLine(ColoredStringBuilder.ToString()); CompleteAnswer += "\r\n"; //CompleteAnswer = builder.ToString(); ClearContext(); if (false) { //string InferPrompt = "System: I am a law professor at a law school. I will answer your legal questions with the help of using the ONLY the following information within the context the provided documentation. I will only answer with facts grounded in the provided documentation and try to do so in three paragraphs or less. I will only answer your question and will not add any futher information or ask anything in return after giving you the direct answer to your question.\r\n\r\n### Start of Documentation:\r\n" + CompleteAnswer.Replace("'", "`") + "\r\n### End of Documentation\r\nUser: " + userInput; //<|begin_of_text|> string InferPrompt = "<|start_header_id|>system<|end_header_id|>\r\n" + "Cutting Knowledge Date: December 2023\r\n" + "Today Date: 26 Jul 2024\r\n\r\n" + "I am a law professor at a law school. I will answer your legal questions with the help of using ONLY the following information within the context of the provided documentation. I will only answer with facts grounded in the provided documentation and try to do so in three paragraphs or less. I will only answer your question and will not add any further information or ask anything in return after giving you the direct answer to your question.\r\n" + "<|eot_id|><|start_header_id|>user<|end_header_id|>\r\n\r\n" + userInput + "\r\n" + "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\r\n\r\n" + "### Start of Documentation:\r\n" + CompleteAnswer.Replace("'", "`") + "\r\n### End of Documentation\r\n"; await foreach (var text in LLMStatelessExecutor.InferAsync(InferPrompt, inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); CompleteAnswer += text; ChatResponseToMeasure += text; } } else { chatHistory.AddMessage(AuthorRole.System, "I am a law professor at a law school. I will answer your legal questions with the help of using the ONLY the following information within the context the provided documentation. I will only answer with facts grounded in the provided documentation and try to do so in three paragraphs or less. I will only answer your question and will not add any futher information or ask anything in return after giving you the direct answer to your question.\r\n\r\n### Start of Documentation:\r\n" + CompleteAnswer.Replace("'", "`") + "\r\n### End of Documentation\r\n\r\nIf I cannot answer I will say 'I am not sure'.\r\n"); chatHistory.AddMessage(AuthorRole.Assistant, "I will answer all your legal questions in context of the law. I will answer your question and will not add anything else to the end of the answer. What legal question may I answer for you?"); await foreach (var text in LLMChatSession.ChatAsync(new ChatHistory.Message(AuthorRole.User, userInput), inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); CompleteAnswer += text; ChatResponseToMeasure += text; } } ClearContext(); } else { await foreach (var text in LLMChatSession.ChatAsync(new ChatHistory.Message(AuthorRole.User, userInput), inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); CompleteAnswer += text; ChatResponseToMeasure += text; } } long TokensInResponse = model.Tokenize(ChatResponseToMeasure, true, false, Encoding.ASCII).Length; Debug.WriteLine("Response Length: " + TokensInResponse); if (!ChatResponseToMeasure.EndsWith("User:") || TokensInResponse > inferenceParams.MaxTokens) { Console.Write("\r\nUser:"); } if (CompleteAnswer != null) { TokensInMemory += model.Tokenize(CompleteAnswer, true, false, Encoding.ASCII).Length; Debug.WriteLine("User added " + model.Tokenize(userInput, true, false, Encoding.ASCII).Length + " Response added: " + model.Tokenize(CompleteAnswer, true, false, Encoding.ASCII).Length + " tokens" + "\r\nTokensInMemory: " + TokensInMemory + "/" + context.ContextSize + "/" + model.ContextSize); if (webSocketContext != null) { if (webSocketContext.WebSocket.State == WebSocketState.Open) { //HttpUtility.HtmlEncode( foreach (var MyString in CompleteAnswer.Split("\r\n")) { webSocketContext.WebSocket.SendAsync(Encoding.UTF8.GetBytes(MyString), WebSocketMessageType.Text, true, CancellationToken.None); } } } Matches = Regex.Match(CompleteAnswer, @"myArray = \[.*?\];"); if (Matches.Success) { Debug.WriteLine(Matches.Value); } } CompleteAnswer = null; Debug.WriteLine("ProcessAIRequest Finished"); } async void StartWebSocketServer() { HttpListener httpListener = new HttpListener(); httpListener.Prefixes.Add("http://localhost:5000/ws/"); httpListener.Start(); Console.WriteLine("WebSocket server started at ws://localhost:5000/ws/"); while (true) { HttpListenerContext httpContext = await httpListener.GetContextAsync(); if (httpContext.Request.IsWebSocketRequest) { webSocketContext = await httpContext.AcceptWebSocketAsync(subProtocol: null); Console.WriteLine("WebSocket connection established."); await ProcessWebSocketMessages(webSocketContext.WebSocket); } else { httpContext.Response.StatusCode = 400; httpContext.Response.Close(); Console.WriteLine("Non-WebSocket request received and rejected."); } } } async Task ProcessWebSocketMessages(WebSocket webSocket) { var buffer = new byte[1024 * 4]; while (webSocket.State == WebSocketState.Open) { WebSocketReceiveResult result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None); if (result.MessageType == WebSocketMessageType.Close) { await webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing", CancellationToken.None); Console.WriteLine("WebSocket connection closed."); } else { string message = Encoding.UTF8.GetString(buffer, 0, result.Count); Console.WriteLine($"{message}"); ProcessAIRequest(message); } } } if (true) { } else { chatHistory.AddMessage(AuthorRole.System, "Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is Lawyer and is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision. Bob will only answer from information provided by Users input and will answer questions as short and precise as possible. After each answer I will cite the exact text that I pulled the inforamtion from word for word."); var Input = File.ReadAllText("D:\\Output-WC 25896.txt"); Console.WriteLine("Console writeline length: " + Input.Length); chatHistory.AddMessage(AuthorRole.User, Input.Substring(0, 10000)); //10000 chatHistory.AddMessage(AuthorRole.Assistant, "Thank you, I will answer your questions only using the text given from User's input."); //chatHistory.AddMessage(AuthorRole.User, Input.Substring(0, 10000)); //10000 //chatHistory.AddMessage(AuthorRole.Assistant, "Thank you, I will answer your questions only using the text given from User's input."); } static void FocusProcess() { IntPtr hWnd; //change this to IntPtr Process[] processRunning = Process.GetProcesses(); foreach (Process pr in processRunning) { if (pr.ProcessName == "notepad") { hWnd = pr.MainWindowHandle; //use it as IntPtr not int ShowWindow(hWnd, 3); SetForegroundWindow(hWnd); //set to topmost } } } StartWebSocketServer(); FocusProcess(); Console.ForegroundColor = ConsoleColor.Yellow; Console.Write("The chat session has started.\nUser: "); Console.ForegroundColor = ConsoleColor.White; void MeasureChatHistoryTokens() { foreach (ChatHistory.Message Value in LLMChatSession.History.Messages.ToArray()) { TokensInMemory += model.Tokenize(Value.Content, true, false, Encoding.ASCII).Length; } } Debug.WriteLine("Starting with tokens" + "\r\nTokensInMemory: " + TokensInMemory + "/" + context.ContextSize + "/" + model.ContextSize); string userInput = ""; while (userInput != "exit") { userInput = Console.ReadLine() ?? ""; ProcessAIRequest(userInput); } async Task RunLLava() { string MultiModel = @"D:\llava-v1.6-mistral-7b.Q4_K_M.gguf"; //string multiModalProj = @"D:\mmproj-mistral7b-f16-q6_k.gguf"; string multiModalProj = @"D:\mmproj-mistral7b-f16.gguf"; string modelImage = @"D:\Ticket4.jpeg"; const int maxTokens = 2048; var prompt = $"{{{modelImage}}}\nUSER:\nProvide a full description of the image, its fields and the values in each field.\nASSISTANT:\n"; var parameters = new ModelParams(MultiModel); using var model = LLamaWeights.LoadFromFile(parameters); using var context = model.CreateContext(parameters); //https://platform.mindee.com/keld/traffic_ticket/live-interface#tab=%22extractedFields%22 //https://www.cnblogs.com/gmmy/p/18146496 //Multimodal model: llava-v1.6-mistral-7b.Q4_K_M.gguf Download address: https://huggingface.co/mradermacher/llava-v1.6-mistral-7b-GGUF/resolve/main/llava-v1.6-mistral-7b.Q4_K_M.gguf //Llava CLIP model: mmproj-mistral7b-f16-q6_k.gguf Download address: https://huggingface.co/cmp-nct/llava-1.6-gguf/resolve/main/mmproj-mistral7b-f16-q6_k.gguf?download=true // Llava Init using var clipModel = LLavaWeights.LoadFromFile(multiModalProj); var ex = new InteractiveExecutor(context, clipModel); Console.ForegroundColor = ConsoleColor.Yellow; Console.WriteLine("The executor has been enabled. In this example, the prompt is printed, the maximum tokens is set to {0} and the context size is {1}.", maxTokens, parameters.ContextSize); Console.WriteLine("To send an image, enter its filename in curly braces, like this {c:/image.jpg}."); var inferenceParams = new InferenceParams() { Temperature = 0.1f, AntiPrompts = new List<string> { "\nUSER:" }, MaxTokens = maxTokens }; do { // Evaluate if we have images // var imageMatches = Regex.Matches(prompt, "{([^}]*)}").Select(m => m.Value); var imageCount = imageMatches.Count(); var hasImages = imageCount > 0; if (hasImages) { var imagePathsWithCurlyBraces = Regex.Matches(prompt, "{([^}]*)}").Select(m => m.Value); var imagePaths = Regex.Matches(prompt, "{([^}]*)}").Select(m => m.Groups[1].Value).ToList(); List<byte[]> imageBytes; try { imageBytes = imagePaths.Select(File.ReadAllBytes).ToList(); } catch (IOException exception) { Console.ForegroundColor = ConsoleColor.Red; Console.Write($"Could not load your {(imageCount == 1 ? "image" : "images")}:"); Console.Write($"{exception.Message}"); Console.ForegroundColor = ConsoleColor.Yellow; Console.WriteLine("Please try again."); break; } // Each prompt with images we clear cache // When the prompt contains images we clear KV_CACHE to restart conversation // See: // https://github.com/ggerganov/llama.cpp/discussions/3620 ex.Context.NativeHandle.KvCacheRemove(LLamaSeqId.Zero, -1, -1); int index = 0; foreach (var path in imagePathsWithCurlyBraces) { // First image replace to tag <image, the rest of the images delete the tag prompt = prompt.Replace(path, index++ == 0 ? "<image>" : ""); } Console.ForegroundColor = ConsoleColor.Yellow; Console.WriteLine($"Here are the images, that are sent to the chat model in addition to your message."); Console.WriteLine(); /* foreach (var consoleImage in imageBytes?.Select(bytes => new CanvasImage(bytes))) { consoleImage.MaxWidth = 50; Spectre.Console.AnsiConsole.Write(consoleImage); } */ Console.WriteLine(); Console.ForegroundColor = ConsoleColor.Yellow; Console.WriteLine($"The images were scaled down for the console only, the model gets full versions."); Console.WriteLine($"Write /exit or press Ctrl+c to return to main menu."); Console.WriteLine(); // Initialize Images in executor // foreach (var image in imagePaths) { ex.Images.Add(await File.ReadAllBytesAsync(image)); } } Console.ForegroundColor = ConsoleColor.White; await foreach (var text in ex.InferAsync(prompt, inferenceParams)) { Console.Write(text); } Console.Write(" "); Console.ForegroundColor = ConsoleColor.Green; prompt = Console.ReadLine(); Console.WriteLine(); // let the user finish with exit // if (prompt != null && prompt.Equals("/exit", StringComparison.OrdinalIgnoreCase)) break; } while (true); } //Unable to use to do "Top level statements must procede namespace and type declerations. [Flags] enum MyLoadedMethod : ushort { Plain = 0, RAG = 1 } |
Here is a sample HTML5 socket webpage if you’d like to access it from a remote machine.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>WebSocket Terminal</title> <style> body { font-family: Arial, sans-serif; background-color: #333; color: #fff; margin: 0; padding: 20px; display: flex; flex-direction: column; height: 100vh; } #terminal { border: 1px solid #555; padding: 10px; overflow-y: auto; background-color: #000; } #input { display: flex; } #messageInput { flex: 1; padding: 10px; border: 1px solid #555; background-color: #222; color: #fff; border-radius: 3px; margin-right: 10px; } #sendButton { padding: 10px; border: 1px solid #555; background-color: #555; color: #fff; border-radius: 3px; cursor: pointer; } </style> </head> <body> <div id="terminal"></div> <div id="input"> <input type="text" id="messageInput" placeholder="Type a message..."> <button id="sendButton">Send</button> </div> <script> const ws = new WebSocket('ws://192.168.0.199:5000/ws/'); const terminal = document.getElementById('terminal'); const messageInput = document.getElementById('messageInput'); const sendButton = document.getElementById('sendButton'); ws.onopen = () => { addMessage('Connected to WebSocket server.', true); }; ws.onmessage = (event) => { addMessage(event.data, false); }; ws.onclose = () => { addMessage('Disconnected from WebSocket server.', true); }; sendButton.addEventListener('click', () => { const message = messageInput.value; if (message) { ws.send(message); addMessage(message, true); messageInput.value = ''; } }); messageInput.addEventListener('keydown', (event) => { if (event.key === 'Enter') { sendButton.click(); } }); function addMessage(message, isSent) { const messageElement = document.createElement('div'); messageElement.textContent = message; // Add a newline only for received messages if (!isSent) { messageElement.style.whiteSpace = 'pre'; } terminal.appendChild(messageElement); terminal.scrollTop = terminal.scrollHeight; } </script> </body> </html> |
Getting IIS Express to listen on a network LAN IP with visual studio.
Normally my post arnt so short but this burned 45 minutes of my time as I struggled to get a native IIS express deployment from VS2022 to use a lan port for debugging remote connections.
Navigate to \RootProjectDirectory\.vs\{ProjectName}\config\applicationhost.config
Run Visual Studio as Administrator, this is critical.
Delete the file after backing it up, then while in Visual studio click Create Virtual Directory using a different port on HTTP then the one you wish to open on lan
Start the project, then Stop debugging. Once Stopped, confirm IIS is no longer running on the taskbar.
Open the newly generated applicationhost.config file within the config directory and add the new binding, mine existed on line 168.
<binding protocol=”http” bindingInformation=”*:44392:{PUT LAN IP HERE}” />
An example of the config will look like this
<sites>
<site name=”WebSite1″ id=”1″ serverAutoStart=”true”>
<application path=”/”>
<virtualDirectory path=”/” physicalPath=”%IIS_SITES_HOME%\WebSite1″ />
</application>
<bindings>
<binding protocol=”http” bindingInformation=”:8080:localhost” />
</bindings>
</site>
<site name=”ProjectName” id=”2″>
<application path=”/” applicationPool=”Clr4IntegratedAppPool”>
<virtualDirectory path=”/” physicalPath=”C:\Users\User\source\repos\ProjectName\ProjectName” />
</application>
<bindings>
<binding protocol=”http” bindingInformation=”*:44391:localhost” />
<binding protocol=”http” bindingInformation=”*:44392:192.168.0.199″ />
</bindings>
</site>
<siteDefaults>
<!– To enable logging, please change the below attribute “enabled” to “true” –>
<logFile logFormat=”W3C” directory=”%AppData%\Microsoft\IISExpressLogs” enabled=”false” />
<traceFailedRequestsLogging directory=”%AppData%\Microsoft” enabled=”false” maxLogFileSizeKB=”1024″ />
</siteDefaults>
<applicationDefaults applicationPool=”Clr4IntegratedAppPool” />
<virtualDirectoryDefaults allowSubDirConfig=”true” />
</sites>
Save, then click Start debug within VS. You may now HTTP from your other machine using the newly added port.
Using LLamaSharp with RAG.
So, The data appears to take 6x-7x the amount of space to archive into the SQLlite memory storage.
Clearing history with a Chatsession does not appear to work correctly so I swapped over to a stateless built which helped so long that the RAG only needs to be called once per question. If additional questions need to be asked on the same data then optimization can be added to speed things up.
I will node that the prompt I’m using to search on provided much better results then the format documented on hugging face oddly enough. “Assistant:” {Query} “User:” instead of “instruct”, “query”.
I also used a function to pull filler words to speed up and create a more reliable keyword function, but the LLM itself wasn’t too bad.
Ultimately, Ill have to stick with the statelessexecutor until I can validate that the ChatSession operates correctly. Once I call LLMSession.load(CleanSession), readding in the prompts appears to not work correctly. During debugging, upon the second add, the count of the ChatSession remained zero.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 |
// See https://aka.ms/new-console-template for more information //leaderboards //https://huggingface.co/spaces/mteb/leaderboard //Implement with Python //https://www.llamaindex.ai/ //https://docs.llamaindex.ai/en/stable/getting_started/discover_llamaindex/ //Install-Package LLamaSharp //Install-Package LLamaSharp.Backend.Cuda11 //https://dotnet.microsoft.com/en-us/download/dotnet/8.0 //https://dotnet.microsoft.com/en-us/download/dotnet //The top line returns waaayy better results then the bottom //Assistant: discovery\r\nUser: //discovery using LLama.Common; using LLama; #pragma warning disable SKEXP0050, SKEXP0001, SKEXP0020 //RAG //Install-Package LLamaSharp.Semantic-Kernel //https://github.com/SciSharp/LLamaSharp/issues/648 using LLamaSharp.SemanticKernel.TextEmbedding; //Install-Package Microsoft.SemanticKernel using Microsoft.SemanticKernel.Connectors; using Microsoft.SemanticKernel.Memory; using Microsoft.SemanticKernel.Text; //Install-Package Microsoft.SemanticKernel.Connectors.Sqlite -Version 1.18.0-alpha using Microsoft.SemanticKernel.Connectors.Sqlite; // using System.IO; using System.Text.RegularExpressions; using System.Diagnostics; using System.Net.WebSockets; using System.Text; using System.Net; using System.Management; using SQLitePCL; using System; using System.Collections; using System.Windows.Documents; using System.Runtime.InteropServices; using static System.Net.Mime.MediaTypeNames; using System.CodeDom.Compiler; using System.Reflection; using Microsoft.Data.Sqlite; using static System.Collections.Specialized.BitVector32; [DllImport("user32.dll")] static extern IntPtr SetForegroundWindow(IntPtr hWnd); [DllImport("user32.dll")] static extern bool ShowWindow(IntPtr hWnd, int nCmdShow); //ShowWindow needs an IntPtr Console.ForegroundColor = ConsoleColor.White; HttpListenerWebSocketContext webSocketContext = null; string RagModelPath = @"D:\bge-large-en-v1.5-q8_0.gguf"; string modelPath = @"D:\Meta-Llama-3.1-8B-Instruct-Q6_K.gguf"; //string modelPath = @"D:\Meta-Llama-3.1-8B-Instruct-Q8_0.gguf"; // change it to your own model path. //string modelPath = @"D:\Meta-Llama-3.1-70B-Instruct-Q6_K-00001-of-00002.gguf"; // Way too large for my computer //var embeddingModelPath = Path.GetFullPath("<path to embed model>/nomic-embed.gguf"); string LocalMemoryPath = @"C:\Users\User\Desktop\MyMemory.db"; string PathToFilesToIngest = "D:\\Embeddings"; ManagementObjectSearcher searcher = new ManagementObjectSearcher("SELECT * FROM Win32_Processor"); List<int> coresCPUs = new List<int>(); foreach (ManagementObject mo in searcher.Get()) { coresCPUs.Add(Convert.ToInt32(mo.Properties["NumberOfCores"].Value)); Debug.WriteLine(Convert.ToInt32(mo.Properties["NumberOfCores"].Value)); } //RAG variables ModelParams RAGparameters = null; ISemanticTextMemory MyMemoryStore = null; //All operations LLamaWeights model = null; LLamaContext context = null; InteractiveExecutor executor = null; var MyStartupMethod = MyLoadedMethod.RAG; string collectionName = "TestCollection"; // Add chat histories as prompt to tell AI how to act. var chatHistory = new ChatHistory(); var chatHistory2 = new ChatHistory(); string Str = null; switch ("rag") { case "train": chatHistory.AddMessage(AuthorRole.System, "Generate new examples based on user input examples, once asked to write additional samples, take the previous request and continue to generate addtional samples. All answers need to strictly be formed into a statement and then answered as a JSON array of strings, always add a semicolon after each string array to close the statement properly. All commands should be for use by the windows operating system. Do not explain, add notes or say anything else unless specifically asked to for any reason."); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate to program files in windows, find a folder with the word steam in it then create a subfolder inside with two text files and populate a random string in each text file. Write the commands using a json array of strings to be processed by an interrupter."); //10000 Str = "myArray = [\"CD \\\\\", \"CD C:\\\\Program Files\\\\\", \"DIR /OD \\\"*steam*\\\"\", \"MKDIR \\\"SteamFolder\\\"\", \"MD \\\"SteamFolder\\\\subfolder\\\", \\\"ECHO New content 1 > \\\"SteamFolder\\\\subfolder\\\\file1.txt\", \"COPY \\\"asdawdafag\\\" > \\\"SteamFolder\\\\subfolder\\\\file2.txt\\\"\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate list the files and folders in the C: drive, the shutdown the computer in 60 seconds."); //10000 Str = "myArray = [\"DIR C:\\\", \"shutdown -s -t 60\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "Great, now continue to write addtional samples, remember to keep the format as a statement, then followed by a JSON answer with a semicolon at the end of each JSON answer. I will ask for over 100 samples."); //10000 chatHistory.AddMessage(AuthorRole.Assistant, "Ping google.com and write the the results to a text file then open the program files folder using explorer. myArray = [\"ping google > c:\\Output.txt\", \"Explorer \"%programfiles%\"\"];"); //10000 chatHistory.AddMessage(AuthorRole.User, "y"); //10000 chatHistory.AddMessage(AuthorRole.Assistant, "write a list of commands that searches for files in the D: Drive that start with the name \"test\" and then copy those files to the c: drive if they contain a word \"echo\" inside. myArray = [\"DIR /S /B /A:-D D:\\test*.txt\", \"COPY \\\"D:\\\\test*.txt\\\" C:\\\" /Y\", \"FINDSTR /C:\"\"echo\"\" D:\\\\test*.txt > E:\\\\log.txt\"];"); //10000 chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Use Moveit.exe to move a file named NTFS.mbr localy, then zip the moved file and display its size. myArray = [\"Moveit \\\"NTFS.mbr\\\" C:\", \"7z a ntfs.zip ntfs.mbr\", \"7z l ntfs.zip\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Write a list of commands to direct a LAN IP to a temp directory, download a single file using curl and extract the extracted content into a subfolder named debug. myArray = [\"MKDIR C:\\\\tmp\\\", \"curl \\\"http://192.168.1.100/somefile.zip\\\" -o C:\\\\tmp\", \"7z e C:\\\\tmp\\\\somefile.zip -od C:\\\\tmp\\\\debug\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Create a random 32 character hexadecimal color code, then use the color picker tool in Windows to open up a default browser using this new color as its background. myArray = [\"wmic os get version > C:\\versions.txt\", \"color cc233330cc2233c2\", \"explorer https://www.google.com\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Write a list of commands to create an Ecrypted zip archive and store it in the users desktop directory, than extract this zip file with AES encryption using 7z. myArray = [\"cipher /E /S:C:\\\\Users\\\\username\\\\Documents\" > C:\\UserEnv, \"cipher /D /S:C:\\\\Users\\\\username\\\\Documents\", \"CIPHER /P\"aVeryStrongPassword123\\* \\\"C:\\\\Users\\\\username\\\\Desktop\\\\enc.zip\\\"\", \"7z e -hp\"aVeryStrongPassword123 \\*.zip\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Display user accounts with administrative privileges, and list the user object properties in Windows registry. myArray = [\"wmic useraccount where Enabled=TRUE and Administrator=TRUE get *\", \"reg query HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\\\Winlogon /v DefaultUserName\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Change password for a service called misconfigure-MDViewer, change priority of CpuGroup1; write a list of commands to accomplish these tasks. myArray = [\"net user \\\"misconfigure-MDViewer\\\" changesPassword!!\"], \"wmic cpu Get Name\", \"wmic process where ProcessID=1234 set Priority=5\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Use the Windows Task Scheduler to create a scheduled task to download a simple text file from a remote server every 10 minutes. myArray = [\"schtasks /create /tn \"MyTask\" /tr \"curl http://192.168.1.100/textfile.txt -o C:\\tmp\\mytext.txt\\\"\", \"schtasks /change /tn \"MyTask\\\" /st 00:00 /ri 10\"];"); chatHistory.AddMessage(AuthorRole.User, "y"); chatHistory.AddMessage(AuthorRole.Assistant, "Write a batch file that opens up Internet Explorer with two tabs one with the url google.com and another with bing.com. myArray = [\"start iexplore.exe \\\"http://www.google.com\\\", start iexplore.exe \\\"http://www.bing.com\\\"\"];"); break; case "production": chatHistory.AddMessage(AuthorRole.System, "Transcribe request from the user into an array of JSON strings labeled 'myArray' to be processed by an interpreter that will parse an array of strings and execute them on a system. All answers need to strictly be formed into a JSON array of strings for the windows operating system. Only give the results in JSON format and do not explain or say anything else unless asked specifically."); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate to program files in windows, find a folder with the word steam in it then create a subfolder inside with two text files and populate a random string in each text file. Write the commands using a json array of strings to be processed by an interrupter."); //10000 Str = "myArray = [\"CD \\\\\", \"CD C:\\\\Program Files\\\\\", \"DIR /OD \\\"*steam*\\\"\", \"MKDIR \\\"SteamFolder\\\"\", \"MD \\\"SteamFolder\\\\subfolder\\\", \\\"ECHO New content 1 > \\\"SteamFolder\\\\subfolder\\\\file1.txt\", \"COPY \\\"asdawdafag\\\" > \\\"SteamFolder\\\\subfolder\\\\file2.txt\\\"\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that will navigate list the files and folders in the C: drive, the shutdown the computer in 60 seconds."); //10000 Str = "myArray = [\"DIR C:\\\", \"shutdown -s -t 60\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "Ping google.com and write the the results to a text file then open the program files folder using explorer."); //10000 Str = "myArray = [\"ping google > c:\\Output.txt\", \"Explorer \"%programfiles%\"\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); chatHistory.AddMessage(AuthorRole.User, "write a list of commands that searches for files in the D: Drive that start with the name \"test\" and then copy those files to the c: drive if they contain a word \"echo\" inside."); //10000 Str = "myArray = [\"DIR /S /B /A:-D D:\\test*.txt\", \"COPY \\\"D:\\\\test*.txt\\\" C:\\\" /Y\", \"FINDSTR /C:\"\"echo\"\" D:\\\\test*.txt > E:\\\\log.txt\"];"; chatHistory.AddMessage(AuthorRole.Assistant, Str); break; case "rag": //chatHistory.AddMessage(AuthorRole.System, "You are a legal assistance. You will be provided context to some data and will answer any questions you receive the best you can with the context provided and do not include information outside of the information provided."); break; default: Console.WriteLine("Number is not between 1 and 4"); break; } //LLM variables var LLMparameters = new ModelParams(modelPath) { ContextSize = 13107, // Does not require alot of memory //ContextSize = 131072, // The longest length of chat as memory. This can bring the computer to a crawl //If commented out, swaps to CPU //GpuLayerCount = 0 GpuLayerCount = 33 // How many layers to offload to GPU. Please adjust it according to your GPU memory. }; LLamaWeights LLMmodel = null; LLamaContext LLMcontext = null; LLamaContext.State CleanContext = null; InteractiveExecutor LLMexecutor = null; StatelessExecutor LLMStatelessExecutor = null; //Not currently in use. Results were bad. StatefulExecutorBase.ExecutorBaseState CleanExectuor = null; ChatSession LLMChatSession = null; SessionState CleanChatSession = null; InferenceParams inferenceParams = new InferenceParams() { Temperature = 0.4f, MaxTokens = 512, // No more than 256 tokens should appear in answer. Remove it if antiprompt is enough for control. AntiPrompts = new List<string> { "\nUser:" } // Stop generation once antiprompts appear. }; async Task LoadModule() { if (MyStartupMethod == MyLoadedMethod.RAG) //RAG //Checks for enum need to be wrapped in a function to be valid. { RAGparameters = new ModelParams(RagModelPath) { ContextSize = 32000, // Does not require alot of memory //ContextSize = 131072, // The longest length of chat as memory. This can bring the computer to a crawl Embeddings = true, //GpuLayerCount = 0 //If commented out, swaps to CPU GpuLayerCount = 33 // How many layers to offload to GPU. Please adjust it according to your GPU memory. }; model = LLamaWeights.LoadFromFile(RAGparameters); context = model.CreateContext(RAGparameters); executor = new InteractiveExecutor(context); var embedder = new LLamaEmbedder(model, RAGparameters); var service = new LLamaSharpEmbeddingGeneration(embedder); MyMemoryStore = new MemoryBuilder().WithMemoryStore(await SqliteMemoryStore.ConnectAsync(LocalMemoryPath)).WithTextEmbeddingGeneration(service).Build(); await IngestFile(); LLMmodel = LLamaWeights.LoadFromFile(LLMparameters); LLMStatelessExecutor = new StatelessExecutor(LLMmodel, LLMparameters); LLMcontext = LLMmodel.CreateContext(LLMparameters); LLMexecutor = new InteractiveExecutor(LLMcontext); LLMChatSession = new(LLMexecutor, chatHistory); } else { LLMmodel = LLamaWeights.LoadFromFile(LLMparameters); LLMcontext = LLMmodel.CreateContext(LLMparameters); LLMexecutor = new InteractiveExecutor(LLMcontext); LLMChatSession = new(LLMexecutor, chatHistory); } CleanExectuor = LLMexecutor.GetStateData(); CleanChatSession = LLMChatSession.GetSessionState(); CleanContext = LLMcontext.GetState(); //_memoryStore //var field = MyMemoryStore.GetType().GetField("_storage", BindingFlags.NonPublic | BindingFlags.Instance); //var memoryStore = (Dictionary<string, string>)field.GetValue(MyMemoryStore); //var keys = memoryStore.Keys.ToList(); using (var connection = new SqliteConnection("Data Source=" + LocalMemoryPath)) { connection.Open(); var command = connection.CreateCommand(); command.CommandText = @"SELECT COUNT(1) from SKMemoryTable"; Debug.WriteLine(command.ExecuteScalar()); //collection, key, metadata, embedding, timestamp //metadata holds a json string of "is_reference", "external_source_name", "id" and "description", "text", "additional_metadata" command.CommandText = @"SELECT DISTINCT key FROM SKMemoryTable"; // Replace 'MemoryTable' with the actual table name using (var reader = command.ExecuteReader()) { while (reader.Read()) { //Console.WriteLine(reader.GetString(0)); } } } //System.Diagnostics.Debugger.Break(); //LLMsession.SaveSession(); } await LoadModule(); string CompleteAnswer = null; Match Matches; long TokensInMemory = 0; async Task IngestFile() { long TotalParagraphsLoaded = 0; Console.WriteLine("===== INGESTING ====="); IList<string> collections = await MyMemoryStore.GetCollectionsAsync(); string folderPath = Path.GetFullPath(PathToFilesToIngest); string[] files = Directory.GetFiles(folderPath); if (collections.Contains(collectionName)) { Console.WriteLine("Found database"); } else { List<Task> tasks = new List<Task>(); foreach (var item in files.Select((path, index) => new { path, index })) { Console.WriteLine($"Ingesting file #{item.index}"); string text = File.ReadAllText(item.path); if (false) { var sentences = TextChunker.SplitPlainTextLines(text, 256); var paragraphs = TextChunker.SplitPlainTextParagraphs(sentences, 1024); foreach (var para in paragraphs.Select((text, index) => new { text, index })) await MyMemoryStore.SaveInformationAsync(collectionName, para.text, $"Document {item.path}, Paragraph {para.index}"); } else if (true) { string[] splitQuestions = Regex.Split(text, @"(?=Question\s\d+\.)"); long ParaIndex = 1; foreach (string question in splitQuestions) { if (!string.IsNullOrWhiteSpace(question)) { try { //description is stored in the metadata column, same with additional_metadata. Remember, converting to weights takes about 6-7x times the space in total. await MyMemoryStore.SaveInformationAsync(collectionName, question.Trim(), ($"Document {item.path}, Paragraph {ParaIndex++}")); } catch { } //tasks.Add(Task.Run(() => memory.SaveInformationAsync(collectionName, question.Trim(), $"Document {item.path}, Paragraph {ParaIndex++}"))); TotalParagraphsLoaded++; } } } else if (false) { string[] splitQuestions = Regex.Split(text, @"(?<=[.!?])\s+|(?<=\.\.)\s*|\r\n|\n"); //each sentence long ParaIndex = 1; foreach (string question in splitQuestions) { if (!string.IsNullOrWhiteSpace(question)) { //await memory.SaveReferenceAsync() await MyMemoryStore.SaveInformationAsync(collectionName, question.Trim(), $"Document {item.path}, Paragraph {ParaIndex++}"); //tasks.Add(Task.Run(() => memory.SaveInformationAsync(collectionName, question.Trim(), $"Document {item.path}, Paragraph {ParaIndex++}"))); TotalParagraphsLoaded++; } } } //await Task.WhenAll(tasks); } Console.WriteLine("Generated database with " + TotalParagraphsLoaded + " items."); } Console.WriteLine("===== DONE INGESTING ====="); } void ClearContext() { return; LLMChatSession.LoadSession(CleanChatSession); Debug.WriteLine(LLMChatSession.HistoryTransform.HistoryToText(LLMChatSession.History)); return; LLMcontext.Dispose(); LLMexecutor = null; LLMChatSession = null; LLMcontext = LLMmodel.CreateContext(LLMparameters); LLMexecutor = new InteractiveExecutor(LLMcontext); LLMChatSession = new(LLMexecutor, chatHistory); return; chatHistory.Messages.Clear(); LLMcontext.LoadState(CleanContext); LLMChatSession.LoadSession(CleanChatSession); LLMexecutor.LoadState(CleanExectuor); return; while (LLMChatSession.History.Messages.Count > 0) { LLMChatSession.RemoveLastMessage(); } return; } string Keywords(string sentence) { HashSet<string> StopWords = new HashSet<string>(new[] { "a", "an", "and", "the", "is", "in", "on", "at", "of", "for", "to", "with", "by", "from", "that", "this", "it", "as", "are", "was", "were", "be", "been", "has", "have", "had", "but", "or", "not", "so", "if", "then", "i", "me", "my", "myself", "we", "our", "ours", "ourselves", "you", "your", "yours", "yourself", "yourselves", "he", "him", "his", "himself", "she", "her", "hers", "herself", "its", "itself", "they", "them", "their", "theirs", "themselves", "what", "which", "who", "whom", "these", "those", "am", "being", "do", "does", "did", "doing", "because", "until", "while", "about", "against", "between", "into", "through", "during", "before", "after", "above", "below", "up", "down", "out", "off", "over", "under", "again", "further", "once", "here", "there", "when", "where", "why", "how", "all", "any", "both", "each", "few", "more", "most", "other", "some", "such", "no", "nor", "only", "own", "same", "than", "too", "very", "s", "t", "can", "will", "just", "don", "should", "now" }); string cleanSentence = Regex.Replace(sentence.ToLower(), @"[^\w\s]", ""); // Split sentence into words var words = cleanSentence.Split(new[] { ' ', '\t', '\n', '\r' }, StringSplitOptions.RemoveEmptyEntries); // Filter out stop words and return remaining words as keywords var keywords = words.Where(word => !StopWords.Contains(word)).ToList(); // Join the keywords into a single string separated by spaces return string.Join(" ", keywords); } async void ProcessAIRequest(string userInput) { Debug.WriteLine("ProcessAIRequest starting"); TokensInMemory += model.Tokenize(userInput, true, false, Encoding.ASCII).Length; //executor.rag //RAG and context building: https://medium.com/@bruno.facco/rag-from-concept-to-application-using-gpt-llama-index-and-sql-retriever-cfb437da99eb //session. String ChatResponseToMeasure = null; if (MyStartupMethod == MyLoadedMethod.RAG) { StringBuilder ColoredStringBuilder = new(); ColoredStringBuilder.Clear(); List<string> sources = []; var resultsfound = false; double minRelevanceScoreValue = 0.8; Console.WriteLine("=== Extracting Keywords ==="); /* string prompt = "System: Ensure that the results contain ALL of the words from the following user query. The words must appear at least once anywhere in the result, without exception.\r\n The results are in the following format, The question number, the question, possible answers then followed by the explanation. The user will give keywords and return the results if those keywords appear in any of the result, either in the question or the explanation.\r\n" + "Assistant: I will only return results that include every single keyword provided by the user. If even one keyword is missing, the result will not be returned, without exception.\r\n" + "User: " + userInput; */ string prompt = userInput; string keywords = ""; string StrchatHistory = "\r\nUser: What is Congresses Commerce Clause powers?\r\n" + "Assistant: Congress Commerce Clause\r\n" + "User: What time is it in new york?\r\n" + "Assistant: time new york\r\n" + "User: How do you fill up a car tire?\r\n" + "Assistant: fill up car tire\r\n" + "User: How fast can a car go on a freeway?\r\n" + "Assistant: fast car freeway\r\n" + "User: How do you make pancakes in an oven?\r\n" + "Assistant: make pancakes oven\r\n" + "User: How do you write a business plan?\r\n" + "Assistant: write business plan\r\n" + "User: What is the capital of Japan?\r\n" + "Assistant: capital Japan\r\n" + "User: How to set up a new email account?\r\n" + "Assistant: set up email account\r\n" + "User: What are the benefits of meditation?\r\n" + "Assistant: benefits meditation\r\n" + "User: How to make a website responsive?\r\n" + "Assistant: make website responsive\r\n" + "User: What is the difference between Python and Java?\r\n" + "Assistant: difference Python Java\r\n" + "User: How do I replace a car battery?\r\n" + "Assistant: replace car battery\r\n" + "User: What is the tallest mountain in the world?\r\n" + "Assistant: tallest mountain world\r\n" + "User: How can I improve my public speaking skills?\r\n" + "Assistant: improve public speaking skills\r\n" + "User: What is the population of Canada?\r\n" + "Assistant: population Canada\r\n" + "User: How to brew coffee using a French press?\r\n" + "Assistant: brew coffee French press\r\n" + "User: How does a blockchain work?\r\n" + "Assistant: blockchain work\r\n" + "User: What are the side effects of aspirin?\r\n" + "Assistant: side effects aspirin\r\n" + "User: How to start learning data science?\r\n" + "Assistant: start learning data science\r\n" + "User: What is the speed of light?\r\n" + "Assistant: speed of light\r\n" + "User: How do I bake a chocolate cake?\r\n" + "Assistant: bake chocolate cake\r\n" + "User: What are the causes of climate change?\r\n" + "Assistant: causes climate change\r\n" + "User: How to play the guitar for beginners?\r\n" + "Assistant: play guitar beginners\r\n" + "User: What are the symptoms of a common cold?\r\n" + "Assistant: symptoms common cold\r\n" + "User: How to create a budget plan?\r\n" + "Assistant: create budget plan\r\n"; if (false) { if (true) { await foreach (var text in LLMStatelessExecutor.InferAsync("System: Return only the keywords from the following statement and do not provide anything else. Use spaces when outputing the keywords values and do not add anything futher." + StrchatHistory + "User: '" + userInput + "'", inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); keywords += text; } } else { //chatHistory.Messages.Clear(); chatHistory.AddMessage(AuthorRole.System, "I am a search engine assistant. I will only return back keywords from any statement you make and nothing else with no exceptions.\r\n"); chatHistory.AddMessage(AuthorRole.User, "What is Congresses Commerce Clause powers?"); chatHistory.AddMessage(AuthorRole.Assistant, "Congress Commerce Clause"); chatHistory.AddMessage(AuthorRole.User, "What time is it in new york?"); chatHistory.AddMessage(AuthorRole.Assistant, "time new york"); chatHistory.AddMessage(AuthorRole.User, "How do you fill up a car tire?"); chatHistory.AddMessage(AuthorRole.Assistant, "fill up car tire"); chatHistory.AddMessage(AuthorRole.User, "How fast can a car go on a freeway?"); chatHistory.AddMessage(AuthorRole.Assistant, "fast car freeway"); chatHistory.AddMessage(AuthorRole.User, "How do you make pancakes in an oven?"); chatHistory.AddMessage(AuthorRole.Assistant, "make pancakes oven"); chatHistory.AddMessage(AuthorRole.User, "How do you write a business plan?"); chatHistory.AddMessage(AuthorRole.Assistant, "write business plan"); chatHistory.AddMessage(AuthorRole.User, "What is the capital of Japan?"); chatHistory.AddMessage(AuthorRole.Assistant, "capital Japan"); chatHistory.AddMessage(AuthorRole.User, "How to set up a new email account?"); chatHistory.AddMessage(AuthorRole.Assistant, "set up email account"); chatHistory.AddMessage(AuthorRole.User, "What are the benefits of meditation?"); chatHistory.AddMessage(AuthorRole.Assistant, "benefits meditation"); chatHistory.AddMessage(AuthorRole.User, "How to make a website responsive?"); chatHistory.AddMessage(AuthorRole.Assistant, "make website responsive"); chatHistory.AddMessage(AuthorRole.User, "What is the difference between Python and Java?"); chatHistory.AddMessage(AuthorRole.Assistant, "difference Python Java"); chatHistory.AddMessage(AuthorRole.User, "How do I replace a car battery?"); chatHistory.AddMessage(AuthorRole.Assistant, "replace car battery"); chatHistory.AddMessage(AuthorRole.User, "What is the tallest mountain in the world?"); chatHistory.AddMessage(AuthorRole.Assistant, "tallest mountain world"); chatHistory.AddMessage(AuthorRole.User, "How can I improve my public speaking skills?"); chatHistory.AddMessage(AuthorRole.Assistant, "improve public speaking skills"); chatHistory.AddMessage(AuthorRole.User, "What is the population of Canada?"); chatHistory.AddMessage(AuthorRole.Assistant, "population Canada"); chatHistory.AddMessage(AuthorRole.User, "How to brew coffee using a French press?"); chatHistory.AddMessage(AuthorRole.Assistant, "brew coffee French press"); chatHistory.AddMessage(AuthorRole.User, "How does a blockchain work?"); chatHistory.AddMessage(AuthorRole.Assistant, "blockchain work"); chatHistory.AddMessage(AuthorRole.User, "What are the side effects of aspirin?"); chatHistory.AddMessage(AuthorRole.Assistant, "side effects aspirin"); chatHistory.AddMessage(AuthorRole.User, "How to start learning data science?"); chatHistory.AddMessage(AuthorRole.Assistant, "start learning data science"); chatHistory.AddMessage(AuthorRole.User, "What is the speed of light?"); chatHistory.AddMessage(AuthorRole.Assistant, "speed of light"); chatHistory.AddMessage(AuthorRole.User, "How do I bake a chocolate cake?"); chatHistory.AddMessage(AuthorRole.Assistant, "bake chocolate cake"); chatHistory.AddMessage(AuthorRole.User, "What are the causes of climate change?"); chatHistory.AddMessage(AuthorRole.Assistant, "causes climate change"); chatHistory.AddMessage(AuthorRole.User, "How to play the guitar for beginners?"); chatHistory.AddMessage(AuthorRole.Assistant, "play guitar beginners"); chatHistory.AddMessage(AuthorRole.User, "What are the symptoms of a common cold?"); chatHistory.AddMessage(AuthorRole.Assistant, "symptoms common cold"); chatHistory.AddMessage(AuthorRole.User, "How to create a budget plan?"); chatHistory.AddMessage(AuthorRole.Assistant, "create budget plan"); await foreach (var text in LLMChatSession.ChatAsync(new ChatHistory.Message(AuthorRole.User, userInput), inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); keywords += text; } } } keywords = Keywords(userInput); Console.WriteLine(keywords); while (!resultsfound) { Console.WriteLine("=== RETRIEVING [" + minRelevanceScoreValue + "]==="); //"Instruct:Given the query, find all relevant text.\r\nQuery: " + keywords + "\r\n" //"Assistant: " + keywords + "\r\nUser:" //await foreach (var result in MyMemoryStore.SearchAsync(collectionName, "Query: " + keywords + "\r\n", limit: 10, minRelevanceScore: minRelevanceScoreValue)) await foreach (var result in MyMemoryStore.SearchAsync(collectionName, "Assistant: " + keywords + "\r\nUser:", limit: 10, minRelevanceScore: minRelevanceScoreValue)) { resultsfound = true; //builder.AppendLine(result.Metadata.Text + "\r\n\r\n" + result.Relevance + "\r\n\r\n"); //CompleteAnswer += result.Metadata.Text; string highlightedText = result.Metadata.Text; foreach (var word in keywords.Split(' ', StringSplitOptions.RemoveEmptyEntries)) { highlightedText = Regex.Replace(highlightedText, " " + word, match => $"\x1b[31m{match.Value}\x1b[0m", RegexOptions.IgnoreCase); } CompleteAnswer += result.Metadata.Text + "\r\n"; ColoredStringBuilder.AppendLine(highlightedText + "\r\n\r\n" + result.Relevance + "\r\n\r\n"); sources.Add(result.Metadata.Id); } minRelevanceScoreValue -= .05d; if (minRelevanceScoreValue < .2d) { Console.WriteLine("Unable to find any subject using your keywords"); return; } } ColoredStringBuilder.AppendLine(""" Sources: """); CompleteAnswer += ("\r\n\r\nSources:\r\n"); foreach (string source in sources) { ColoredStringBuilder.AppendLine($" {source}"); CompleteAnswer += $" {source}\r\n"; } CompleteAnswer += "\r\n"; Console.WriteLine("=== DONE RETRIEVING ==="); Console.WriteLine(ColoredStringBuilder.ToString()); CompleteAnswer += "\r\n"; //CompleteAnswer = builder.ToString(); ClearContext(); if (true) { //string InferPrompt = "System: I am a law professor at a law school. I will answer your legal questions with the help of using the ONLY the following information within the context the provided documentation. I will only answer with facts grounded in the provided documentation and try to do so in three paragraphs or less. I will only answer your question and will not add any futher information or ask anything in return after giving you the direct answer to your question.\r\n\r\n### Start of Documentation:\r\n" + CompleteAnswer.Replace("'", "`") + "\r\n### End of Documentation\r\nUser: " + userInput; //<|begin_of_text|> string InferPrompt = "<|start_header_id|>system<|end_header_id|>\r\n" + "Cutting Knowledge Date: December 2023\r\n" + "Today Date: 26 Jul 2024\r\n\r\n" + "I am a law professor at a law school. I will answer your legal questions with the help of using ONLY the following information within the context of the provided documentation. I will only answer with facts grounded in the provided documentation and try to do so in three paragraphs or less. I will only answer your question and will not add any further information or ask anything in return after giving you the direct answer to your question.\r\n" + "<|eot_id|><|start_header_id|>user<|end_header_id|>\r\n\r\n" + userInput + "\r\n" + "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\r\n\r\n" + "### Start of Documentation:\r\n" + CompleteAnswer.Replace("'", "`") + "\r\n### End of Documentation\r\n"; await foreach (var text in LLMStatelessExecutor.InferAsync(InferPrompt, inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); CompleteAnswer += text; ChatResponseToMeasure += text; } } else { chatHistory.AddMessage(AuthorRole.System, "I am a law professor at a law school. I will answer your legal questions with the help of using the ONLY the following information within the context the provided documentation. I will only answer with facts grounded in the provided documentation and try to do so in three paragraphs or less. I will only answer your question and will not add any futher information or ask anything in return after giving you the direct answer to your question.\r\n\r\n### Start of Documentation:\r\n" + CompleteAnswer.Replace("'", "`") + "\r\n### End of Documentation\r\n\r\nIf I cannot answer I will say 'I am not sure'.\r\n"); chatHistory.AddMessage(AuthorRole.Assistant, "I will answer all your legal questions in context of the law. I will answer your question and will not add anything else to the end of the answer. What legal question may I answer for you?"); await foreach (var text in LLMChatSession.ChatAsync(new ChatHistory.Message(AuthorRole.User, userInput), inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); CompleteAnswer += text; ChatResponseToMeasure += text; } } ClearContext(); } else { await foreach (var text in LLMChatSession.ChatAsync(new ChatHistory.Message(AuthorRole.User, userInput), inferenceParams)) // Generate the response streamingly. { Console.ForegroundColor = ConsoleColor.White; Console.Write(text); CompleteAnswer += text; ChatResponseToMeasure += text; } } long TokensInResponse = model.Tokenize(ChatResponseToMeasure, true, false, Encoding.ASCII).Length; Debug.WriteLine("Response Length: " + TokensInResponse); if (!ChatResponseToMeasure.EndsWith("User:") || TokensInResponse > inferenceParams.MaxTokens) { Console.Write("\r\nUser:"); } if (CompleteAnswer != null) { TokensInMemory += model.Tokenize(CompleteAnswer, true, false, Encoding.ASCII).Length; Debug.WriteLine("User added " + model.Tokenize(userInput, true, false, Encoding.ASCII).Length + " Response added: " + model.Tokenize(CompleteAnswer, true, false, Encoding.ASCII).Length + " tokens" + "\r\nTokensInMemory: " + TokensInMemory + "/" + context.ContextSize + "/" + model.ContextSize); if (webSocketContext != null) { if (webSocketContext.WebSocket.State == WebSocketState.Open) { //HttpUtility.HtmlEncode( foreach (var MyString in CompleteAnswer.Split("\r\n")) { webSocketContext.WebSocket.SendAsync(Encoding.UTF8.GetBytes(MyString), WebSocketMessageType.Text, true, CancellationToken.None); } } } Matches = Regex.Match(CompleteAnswer, @"myArray = \[.*?\];"); if (Matches.Success) { Debug.WriteLine(Matches.Value); } } CompleteAnswer = null; Debug.WriteLine("ProcessAIRequest Finished"); } async void StartWebSocketServer() { HttpListener httpListener = new HttpListener(); httpListener.Prefixes.Add("http://localhost:5000/ws/"); httpListener.Start(); Console.WriteLine("WebSocket server started at ws://localhost:5000/ws/"); while (true) { HttpListenerContext httpContext = await httpListener.GetContextAsync(); if (httpContext.Request.IsWebSocketRequest) { webSocketContext = await httpContext.AcceptWebSocketAsync(subProtocol: null); Console.WriteLine("WebSocket connection established."); await ProcessWebSocketMessages(webSocketContext.WebSocket); } else { httpContext.Response.StatusCode = 400; httpContext.Response.Close(); Console.WriteLine("Non-WebSocket request received and rejected."); } } } async Task ProcessWebSocketMessages(WebSocket webSocket) { var buffer = new byte[1024 * 4]; while (webSocket.State == WebSocketState.Open) { WebSocketReceiveResult result = await webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None); if (result.MessageType == WebSocketMessageType.Close) { await webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing", CancellationToken.None); Console.WriteLine("WebSocket connection closed."); } else { string message = Encoding.UTF8.GetString(buffer, 0, result.Count); Console.WriteLine($"{message}"); ProcessAIRequest(message); } } } if (true) { } else { chatHistory.AddMessage(AuthorRole.System, "Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is Lawyer and is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision. Bob will only answer from information provided by Users input and will answer questions as short and precise as possible. After each answer I will cite the exact text that I pulled the inforamtion from word for word."); var Input = File.ReadAllText("D:\\Output-WC 25896.txt"); Console.WriteLine("Console writeline length: " + Input.Length); chatHistory.AddMessage(AuthorRole.User, Input.Substring(0, 10000)); //10000 chatHistory.AddMessage(AuthorRole.Assistant, "Thank you, I will answer your questions only using the text given from User's input."); //chatHistory.AddMessage(AuthorRole.User, Input.Substring(0, 10000)); //10000 //chatHistory.AddMessage(AuthorRole.Assistant, "Thank you, I will answer your questions only using the text given from User's input."); } static void FocusProcess() { IntPtr hWnd; //change this to IntPtr Process[] processRunning = Process.GetProcesses(); foreach (Process pr in processRunning) { if (pr.ProcessName == "notepad") { hWnd = pr.MainWindowHandle; //use it as IntPtr not int ShowWindow(hWnd, 3); SetForegroundWindow(hWnd); //set to topmost } } } StartWebSocketServer(); FocusProcess(); Console.ForegroundColor = ConsoleColor.Yellow; Console.Write("The chat session has started.\nUser: "); Console.ForegroundColor = ConsoleColor.White; void MeasureChatHistoryTokens() { foreach (ChatHistory.Message Value in LLMChatSession.History.Messages.ToArray()) { TokensInMemory += model.Tokenize(Value.Content, true, false, Encoding.ASCII).Length; } } Debug.WriteLine("Starting with tokens" + "\r\nTokensInMemory: " + TokensInMemory + "/" + context.ContextSize + "/" + model.ContextSize); string userInput = ""; while (userInput != "exit") { userInput = Console.ReadLine() ?? ""; ProcessAIRequest(userInput); } //Unable to use to do "Top level statements must procede namespace and type declerations. [Flags] enum MyLoadedMethod : ushort { Plain = 0, RAG = 1 } |
Installing windows 10 on a Mid 2012 Mac Book without Bootcamp assistant
Download Rufus: https://rufus.ie/downloads/
Download windows 10: https://www.microsoft.com/en-us/software-download/windows10 – Create an .ISO to burn with Rufus
When using Rufus to burn the image to the thumbdrive, it is critical that you select MBR partitioning and that BIOS / Legacy mode is selected. WARNING: If this step is not followed and UEFI is used, which is the default for Windows 10 thumb/flash drive deployments, installation and driver support from bootcamp after windows is loaded will fail and you may receive a bluescreen from Intel’s video graphics driver. On top of that the memory allocation will not be in the correct location for DSDT (See links below on how to solve if you cannot reinstall) and your audio will not work out of the box.
Once windows is installed, grab Brigadier.
https://github.com/timsutton/brigadier – Select “Releases” on the right hand side or click here: https://github.com/timsutton/brigadier/releases/
Run Brigadier’s EXE as ADMINISTRATOR and let it run. Ensue the setup located within the BootCamp-041-84821\bootcamp\setup.exe is executed. This process will take about 20-30 minutes to complete as it installs all the stock drivers.
After, run “Apple Software Update”. The first time it may complain about the bootcamp signature not matching but it will still be able to update the “Apple Software Updater” itself. Continue updating and rebooting untill all updates are completed. NOTE: During this process I did receive a hanging white screen once and was required to hold down the power button to hard shutdown upon the boot process and powered it on moments later and the laptop continued to operate as normal.
Exception: During my experience everything installed but the latest “Boot Camp 6 Update for Windows 10”, however, did not impact my deployment at all as my sounds, Wifi and all devices in device driver checked out perfectly fine.
After all updates are completed, move to Windows Update and install all updates and your good to go!
Other links that are note worthy during my review of this issue.
https://discussions.apple.com/thread/250137979?sortBy=rank